Prediction as a candidate for learning deep hierarchical models of data |
Rasmus Berg Palm
|
| Abstract | Recent fi ndings [HOT06] have made possible the learning of deep layered hierarchical representations of data mimicking the brains working. It is hoped that this paradigm will unlock some of the power of the brain and lead to advances towards true AI.
In this thesis I implement and evaluate state-of-the-art deep learning models and using these as building blocks I investigate the hypothesis that predicting the time-to-time sensory input is a good learning objective. I introduce the Predictive Encoder (PE) and show that a simple non-regularized learning rule, minimizing prediction error on natural video patches leads to receptive fi elds similar to those found in Macaque monkey visual area V1. I scale this model to video of natural scenes by introducing the Convolutional Predictive Encoder (CPE) and show similar results. Both models can be used in deep architectures as a deep learning module. |
| Type | Master's thesis [Academic thesis] |
| Year | 2012 |
| Publisher | Technical University of Denmark, DTU Informatics, E-mail: reception@imm.dtu.dk |
| Address | Asmussens Alle, Building 305, DK-2800 Kgs. Lyngby, Denmark |
| Series | IMM-M.Sc.-2012-31 |
| Note | |
| Electronic version(s) | [pdf] |
| Publication link | http://www.imm.dtu.dk/English.aspx |
| BibTeX data | [bibtex] |
| IMM Group(s) | Intelligent Signal Processing |