Prediction as a candidate for learning deep hierarchical models of data

Rasmus Berg Palm

AbstractRecent fi ndings [HOT06] have made possible the learning of deep layered hierarchical representations of data mimicking the brains working. It is hoped that this paradigm will unlock some of the power of the brain and lead to advances towards true AI.
In this thesis I implement and evaluate state-of-the-art deep learning models and using these as building blocks I investigate the hypothesis that predicting the time-to-time sensory input is a good learning objective. I introduce the Predictive Encoder (PE) and show that a simple non-regularized learning rule, minimizing prediction error on natural video patches leads to receptive fi elds similar to those found in Macaque monkey visual area V1. I scale this model to video of natural scenes by introducing the Convolutional Predictive Encoder (CPE) and show similar results. Both models can be used in deep architectures as a deep learning module.
TypeMaster's thesis [Academic thesis]
PublisherTechnical University of Denmark, DTU Informatics, E-mail:
AddressAsmussens Alle, Building 305, DK-2800 Kgs. Lyngby, Denmark
NoteSupervised by Associate Professor Ole Winther,, DTU Informatics, and Morten Mørup,, DTU Informatics
Electronic version(s)[pdf]
Publication link
BibTeX data [bibtex]
IMM Group(s)Intelligent Signal Processing

Back  ::  IMM Publications