Variational Approach to Factor Analysis and Related Models

Frederik Brink Nielsen

AbstractThis thesis focuses on factor analysis, extended factor analysis and parallel factor analysis. For these models Expectation maximization and variational Bayes algorithms are derived. The variational algorithms are implemented with hierarchical automatic relevance determination priors in order to automatically infer the model order. The algorithms are tested on synthetic data and used for analyzing amino acids and fMRI data. In a model selection task the variational lower bound on the marginal likelihood inferred, in a reliable manner, the correct model type. On the other hand, it was found that the lower bound under estimated the model order compared to the Bayesian information criterion. The eŁect of hyperparameter optimiza- tion is discussed in relation to the generalization error. Furthermore, it is found that the variational algorithms have properties similar to independent component analysis. A model called probabilistic partial least squares is proposed and used in a NPAIRS analysis of the fMRI data. The NPAIRS analysis and the parallel factor model were found to give very similar results.
Keywordsvariational Bayes, model selection, hyperparameter optimization, factor analysis, parallel factor analysis, ARD, fMRI
TypeMaster's thesis [Academic thesis]
Year2004
PublisherInformatics and Mathematical Modelling, Technical University of Denmark, DTU
AddressRichard Petersens Plads, Building 321, DK-2800 Kgs. Lyngby
SeriesIMM-Thesis-2004-30
NoteSupervised by Professor Lars Kai Hansen and Associate Professor Ole Winther
Electronic version(s)[pdf] [ps]
BibTeX data [bibtex]
IMM Group(s)Intelligent Signal Processing


Back  ::  IMM Publications