- Beta distribution is the conjugate prior of Bernulli distribution; and Beta distribution is also the conjugate prior of binomial distribution;
- Dirichlet distribution is the conjugate prior of multivariate Bernulli distribution; and Dirichlet distribution is also the conjugate prior of multinomial distribution;
After reading the chapter 2 and Appendix B in Bishop's PRML book again and again, I finally realized one thing today: conjugacy is related to the parameter rather than the distribution and this explains the above puzzles.
See the definition of conjugate prior on wikipedia:
In Bayesian probability theory, a class of prior probability distributions p(θ) is said to be conjugate to a class of likelihood functions p(x|θ) if the resulting posterior distributions p(θ|x) are in the same family as p(θ); the prior and posterior are then called conjugate distributions, and the prior is called a conjugate prior for the likelihood.
Thus,
- Beta(μ|a,b) is the conjugate prior of Bernulli(x|μ) and binomial(m|N,μ) , because in both distributions, μ has a conjugate prior Beta
- Dirichlet distribution is the conjugate prior of multivariate Bernulli Discrete(x_{1:K}|μ) and multinomial(m_{1:K}|μ), because in both distributions, vector μ has a conjugate prior Dirichlet
For example, for Bernulli(x|p), its conjugate prior is Beta(p|a,b), then the posterior distribution of the parameter p after giving n observations is
p(p|x_{1:N}, a, b) = Beta(p|a+n_1, b+n_0), where n_1 and n_0 are the counts of x_i = 1 and x_i = 0 respectively.
But for the example in this website, the poster distribution is for the case of one observation.
No comments:
Post a Comment