Friday, September 4, 2009

Derivation of Gibbs Sampling for Finite Gaussian Mixture Model

Recently, I read Rasmussen's NIPS 2000 paper "Infinite Gaussian Mixture Model". I found there is no much details how to derive the equations in the GMM model. In the process deriving these equations by myself, I found some problems in this paper. His definition of Gamma distribution introduces lots of confusions and his computation of Gamma posterior is doubtful. All these problems make it difficult for others to implement his algorithm in Matlab. So I decide to write down my derivation, together with the equations related to Gamma distribution, which are correct in my opinion. I also derive several standard conjugate distributions, including Gaussian and Gamma , as listed in the Table of conjugate distributions on wikipedia, which are used in my derivation.

I will try to do the Matlab coding later on. 



The PDF file of the report is here.

1 comment: