Mads Nielsen
Visual attention as an information bottleneck
An optimal method of how to sample information from large information sources to obtain as much relevant information using as few samples as possible is derived. The approach is inspired by the statistical technique of importance sampling. Importance sampling is often used in simulation of stochastic processes when the emphasis is on creating a sampling scheme that minimizes the variance of an integral estimator, and special attention must be given to some important but rare events. In this paper, importance sampling is reformulated in the inference setting. The aim is to optimally sample a distribution of a stochastic variable X in order to learn as fast, i.e., with as few samples, as possible about a related variable Y. This strategy may be of interest in many different scenarios such as for searching relevant information in large physical experiments, for guiding robot attention, or any other purposive data mining, where data is abundant, but processing capability limited.