@ARTICLE\{IMM2010-05948,
    author       = "T. J. Abrahamsen and L. K. Hansen",
    title        = "Regularized Pre-image Estimation for Kernel {PCA} De-noising Input Space Regularization and Sparse Reconstruction",
    year         = "2010",
    keywords     = "Kernel {PCA} - Pre-image - Regularization - De-noising - Sparsity",
    journal      = "Journal of Signal Processing Systems",
    volume       = "61",
    editor       = "",
    number       = "",
    publisher    = "Springer New York",
    url          = "http://www2.compute.dtu.dk/pubdb/pubs/5948-full.html",
    abstract     = "The main challenge in de-noising by kernel Principal Component Analysis (PCA) is the mapping of de-noised feature space points back into input space, also referred to as “the pre-image problem”. Since the feature space mapping is typically not bijective, pre-image estimation is inherently illposed. As a consequence the most widely used estimation schemes lack stability. A common way to stabilize such estimates is by augmenting the cost function by a suitable constraint on the solution values. For de-noising applications we here propose Tikhonov input space distance regularization as a stabilizer for pre-image estimation, or sparse reconstruction by Lasso regularization in cases where the main objective is to improve the visual simplicity. We perform extensive experiments on the {USPS} digit modeling problem to evaluate the stability of three widely used pre-image estimators. We show that the previous methods lack stability in the is non-linear regime, however, by applying our proposed input space distance regularizer the estimates are stabilized with a limited sacrifice in terms of de-noising efficiency. Furthermore, we show how sparse reconstruction can lead to improved visual quality of the estimated pre-image."
}