Guillaume Obozinski
Lecture 1 - L1 regularization, formulation, properties and algorithms
The Lasso proposed by Tibshirani in 1996 is based on L1 regularization, which provides a convex optimization approach to variable selection. In this lecture, I will introduce L1 regularization and the associated optimization problems. I will review convex optimization concepts that allow to analyse their solutions. We will then consider several of the most efficient algorithms (coordinate descent, algorithms based on variational forms of the norm and proximal algorithms) to solve the corresponding optimization problem and the statistical properties of the Lasso solutions that are relevant from an applied perspective.
Lecture 2 - Group Sparsity, multiple kernel learning and sparsity for matrices
The group Lasso proposed by Yuan and Lin (2006) generalizes the Lasso to the selection of variables in groups. Its is closely related to Multiple Kernel Learning, a setting where the data is represented implicitly through several positive definite kernels which one would like to combine optimally. In this lecture, I will present the group Lasso, its connection to Multiple Kernel Learning and show how algorithms for the Lasso generalize to this case. In a second part, we will then consider the case of matrices and the different types of sparsity that apply to them, with associated algorithms.
Lecture 3 - Structured Sparsity
The notion of structured sparsity is a notion that emerged recently through the work of several authors. It considers models that are not only sparse, but for which the sparsity pattern is further assumed to have a certain structure or have some form of regularity. Typical examples are provided by regularization with the total variation, and by cases where variables are organised in a hierarchy or on an undirected graph or more generally where the family of sparsity pattern desired is stable by union or intersection. This lecture will provide a tour of recently proposed structured sparse methods, with algorithm and applications, with an emphasis on structured dictionary learning.