Parameter estimation for text analysis
by Gregor Heinrich and
Distributed Gibbs Sampling of Latent Topic Models: The Gritty Details
by Yi Wang now, hoping to grasp the technical details of Gibbs sampling in LDA model. Both are excellent materials for newbies. Gregor's report provide more background knowledge such as Bayesian learning, Bayes network and related distributions. The derivation of Gibbs sampling for LDA only contains the second half of this report. Yi's report expands the derivations of the latter part. So it is better to read Gregor's report before Yi's. I do not know this point and start with Yi's and found it will be easier otherwise.
The following is a summary of the conjugacy of the Bernulli distribution and its multivariate counter part, summarized from Gregor's report.
| bi-variate | multi-variate | |
| single trial |  |  |
| multiple trials, n times |  |  |
| conjugate prior to p |  |  |
| conjugate prior's normalization term |  |  |
| posterior of given n observations |  |  |
| likelihood of n observations in terms of hyprparameter, by integrating out parameter p |  |  |
No comments:
Post a Comment