This thesis investigates various methods for carrying out approximate inference in intractable probabilistic models. By capturing the relationships between random variables, the framework of graphical models hints at which sets of random variables pose a problem to the inferential step. The approximating techniques used in this thesis originate from the field of statistical physics which for decades has been facing the same type of intractable computations when analyzing large systems of interacting variables e.g. magnetic spin systems. In general, these approximating techniques are known as mean field methods.
The thesis provides a brief introduction to the basic methodology of learning and inference in graphical models as well as a short review of the various types of mean field approximations which recently have been shown to be efficient for carrying out approximate inference in intractable probabilistic models.
Starting from the naive mean field approximation we derive for the independent component analysis (ICA) model with instantaneous mixing general expressions for the posterior quantities needed to perform learning by Expectation-Maximization (EM). Furthermore, we explore the feasibility of going beyond the naive mean field approximation for this model. In fact, it turns out that the overcomplete ICA problem can be solved using a simple linear response correction to the mean sufficient statistics obtained by naive mean field approximation. In addition, we apply to the ICA problem an adaptive version of the Thouless, Anderson and Palmer (TAP) mean field approach which is due to Opper and Winther.
To illustrate the methodology on a real world problem, an explorative analysis of a functional magnetic resonance imaging (fMRI) dataset from a visual activation study is carried out using ICA with binary sources. It is shown this approach, which is computationally efficient, infers reasonable brain activation functions.
Finally, we outline various ways of carrying out approximate message passing in probabilistic models for which marginalization over some of the clique variables is intractable.
Resume på dansk:
I nærværende afhandling undersøgers forskellige teknikker til at udføre approximativ inferens i beregningsmæssigt tunge probabilistiske modeller. Med udgangspunkt i teorien om grafiske modeller er det muligt direkte at få et indblik i, hvornår approximerende metoder er påkrævede. De teknikker, som er benyttet i denne afhandling, har alle deres oprindelse i statistisk fysik. I løbet af de sidste årtier har folk indenfor dette område udviklet approximerende metoder for at kunne analysere systemer med mange vekselvirkende enheder som f.eks. magnetiske spin systemer. Disse metoder går samlet under betegnelsen middelfeltsmetoder.
Denne afhandling giver foruden en kort introduktion til inferens og parameterestimation i grafiske modeller også en oversigt over de forskellige metoder, som i tidens løb har vist sig egnede for approximativ inferens i probabilistiske modeller.
Med udgangspunkt i den naive middelfeltsapproximation udleder vi generelle udtryk for kilde-posteriorsandsynligheden i en model for ``independent component analysis'' (ICA) hvor kilderne blandes instantant. For denne model undersøger vi fordelene ved at benytte mere advancerede approximationer. Det viser sig, at det underbestemte tilfælde, hvor der er flere kilder end mikrofoner, kan løses ved en lineær responskorrektion af de sufficiente statistikker, som fås fra den naive middelfeltsapproximation. Endeligt anvender vi til dette ICA problem en adaptiv version af Thouless, Anderson and Palmer (TAP) middelfeltsmetoden, som blev foreslået af Opper og Winther.
Vi illustrerer metoden i en explorativ analyse af en sekvens af dynamiske hjerneskanbilleder optaget under et visuelt aktiveringsstudie. Det er vist, at denne metode, som er beregningsmæssig effektiv, rent faktisk er i stand til at finde plausible hjerneaktiveringsmønstre.
Endelig gives der forslag til, hvordan det er muligt at lave approximativ sekventiel inferens i probabilistiske modeller, hvor eksakt marginalisering over enkelte klikkepotentialer er umuligt.