a good review article for LDA

I happened to find a review article for LDA and its application for text, vision and music.
The link is Latent Dirichlet Allocation for Text, Images, and Music
and the slides is here

They are worth to read carefully.

papers: employing semantic hierarchy in object recognition

Semantic hierarchy could play an important role in object recognition. For example, if we know mini-van is a type of car, and we have already a model for car vs. the rest of the world, then we only need to differentiate mini-van from the car, which reduces lots of work. Similar idea has been noticed by object recognition researchers and there are several papers in recent years:

• Exploiting object hierarchy: Combining models from different category levels, ICCV 2007
• Learning and using taxonomies for fast visual categorization, CVPR 2008
• Unsupervised discovery of visual object class hierarchies, CVPR 2008
• Constructing category hierarchies for visual recognition, ECCV 2008
• Unsupervised learning of visual taxonomies, CVPR 2008
• Semantic Hierarchies for Visual Object Recognition, CVPR 2007
  
• Latent Topic Random Fields: Learning using a taxonomy of labels, CVPR 2008

where some are listed in Trevor's course page.

papers: supervised or discriminative topic model

Topic models are originally designed for topic discovery/clustering, not for classification. To use topic models for classification task, we have modify the structure of the topic model to add the class label and use it to bias the topic discovery process.

the following paper present some supervised/discriminative topic models by machine learning guys:

• MedLDA: Maximum Margin Supervised Topic Models for Regression and Classification, ICML 2009
• DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification, NIIPS 2008
• Labeled LDA: A supervised topic model for credit attribution in multi-labeled corpora, EMNLP 2009
• Supervised topic models, NIPS 2007

some supervised/discriminative topic models by computer vision guys:

• A Bayesian Hierarchical Model for Learning Natural Scene Categories, CVPR 2005
  
• Simultaneous Image Classification and Annotation, CVPR 2009
• Spatially coherent latent topic model for concurrent object segmentation and classification, ICCV 2007
• Towards Total Scene Understanding Classification, Annotation and Segmentation in an Automatic Framework, CVPR 2009
• What, where and who? Classifying events by scene and object recognition, ICCV 2007
• Learning Hierarchical Models of Scenes, Objects, and Parts, ICCV 2005
  

papers: extend topic model to deal with temporal dependency

Using topic models to analyze the tread and change of topics along the time line in a document corpus is definitely a cool idea. It has plenty of potentials in video analysis, human action understanding, etc. The following are a few papers related to this idea:

• Dynamic Topic Models, ICML 2006
• Continuous Time Dynamic Topic Models, UAI 2008
• Hidden Topic Markov Models, AISTATS 2007

non-parametric models:

• an HDP-HMM model is described in Yee Whye Teh's HDP paper
• An HDP-HMM for Systems with State Persistence, ICML 2008
• Infinite Hierarchical Hidden Markov Models. K. Heller, Y.W. Teh and D. Gorur. AISTATS 2009

Here is a good discussion on several infinite HMM:
a blog by Jurgen Van Gael, discussing several infinite HMM

Hierarchical Dirichlet Process: A Gentle Introduction

I wrote up a gentle introduction on HDP based on my two previous post:
Chinese restaurant process and Chinese restaurant franchise
from DP mixture to hierarchical DP mixture

The PDF file is here.

Chinese restaurant process and Chinese restaurant franchise

An illustration of Chinese restaurant process (CRP) and Chinese restaurant franchise (CRF). Materials are from Yee Whye Teh's 2004 technical report on "Hierarchical Dirichlet Process"

Chinese restaurant process (CRP):

For a set of random variables distributed according to , we have the following conditional distribution:

This can be described in a Chinese restaurant process metaphor:

• Consider a Chinese restaurant with an unbounded number of tables, 
• The first customer sits at table 1
• Suppose there are K tables occupied before the i-th customer comes, he can sit at 

            

The relationship between and can be best illustrated as in the following figure.

random variables	meaning	metophor
	random variables	customer i
	distinct values within all	table k
	the number of  associated to	the number of customers sitting around table k

Chinese restaurant franchise (CRF):

An essentially two-level Chinese restaurant process:

• Within a restaurant, customers   choose tables
• Within all restaurants, tables  choose dishes

In both levels, the choosing follows the Chinese restaurant process as illustrated above.

At the restaurant level, customers   choose tables according to the following distribution:

At this level, the metaphor of CRF is the same as the one of CRP as described above, except some changes on symbols

• Consider restaurant j with an unbounded number of tables, 
• The first customer sits at table 1
• Suppose there are  tables occupied before the i-th customer comes, he can sit at 

            

The relationship between and can be best illustrated as in the following figure:

At the franchise level, table   choose dishes according to the following distribution:

This can be described in a Chinese restaurant franchise metaphor:

• Consider a Chinese restaurant franchise, whose J restaurants share a menu with unbounded number of dishes, 
• At each table of each restaurant, one dish is ordered from the public menu by the first customer who sits there, and it is shared among all customers who sit at that table. Multiple tables at multiple restaurants can serve the same dish
  
• Suppose there are  tables occupied before the i-th customer comes restaurant j and there are total K dishes has been ordered among all restaurants in the franchise. He can sit at an occupied table or a new table with certain probability, as described above. If he sits at an occupied table, he shares the dish that has been ordered at that table. If he sits at a new table, he order a dish for that table according to its popularity among the whole franchise, while a new dish can also be tried. 

            

The relationship between and can be best illustrated as in the following figure.

random variables	meaning	metophor
	random variables	customer i in restaurant  j
	distinct values of in group j	table t in restaurant j
	index of associated to ,	the table taken by customer i in restaurantj,.i.e., Table()=, Customer() =
	the number of associated to in group j	the number of customers sitting around table t in restaurantj
	distinct values in within all groups,	dish k, which is shared within all restaurants
	index of associated to ,	the dish ordered by table t in restaurantj, i.e., Dish() = , Table() =
	the number of associated to in group j	the number of tables ordered dish k in restaurantj
	, i.e., the number of associated to over all j	the total number of tables ordered dish k within all restaurants