---
res:
  bibo_abstract:
  - 'Autoencoders are a prominent model in many empirical branches of machine learning
    and lossy data compression. However, basic theoretical questions remain unanswered
    even in a shallow two-layer setting. In particular, to what degree does a shallow
    autoencoder capture the structure of the underlying data distribution? For the
    prototypical case of the 1-bit compression of sparse Gaussian data, we prove that
    gradient descent converges to a solution that completely disregards the sparse
    structure of the input. Namely, the performance of the algorithm is the same as
    if it was compressing a Gaussian source - with no sparsity. For general data distributions,
    we give evidence of a phase transition phenomenon in the shape of the gradient
    descent minimizer, as a function of the data sparsity: below the critical sparsity
    level, the minimizer is a rotation taken uniformly at random (just like in the
    compression of non-sparse data); above the critical sparsity, the minimizer is
    the identity (up to a permutation). Finally, by exploiting a connection with approximate
    message passing algorithms, we show how to improve upon Gaussian performance for
    the compression of sparse data: adding a denoising function to a shallow architecture
    already reduces the loss provably, and a suitable multi-layer decoder leads to
    a further improvement. We validate our findings on image datasets, such as CIFAR-10
    and MNIST.@eng'
  bibo_authorlist:
  - foaf_Person:
      foaf_givenName: Kevin
      foaf_name: Kögler, Kevin
      foaf_surname: Kögler
      foaf_workInfoHomepage: http://www.librecat.org/personId=94ec913c-dc85-11ea-9058-e5051ab2428b
  - foaf_Person:
      foaf_givenName: Aleksandr
      foaf_name: Shevchenko, Aleksandr
      foaf_surname: Shevchenko
      foaf_workInfoHomepage: http://www.librecat.org/personId=F2B06EC2-C99E-11E9-89F0-752EE6697425
  - foaf_Person:
      foaf_givenName: Hamed
      foaf_name: Hassani, Hamed
      foaf_surname: Hassani
  - foaf_Person:
      foaf_givenName: Marco
      foaf_name: Mondelli, Marco
      foaf_surname: Mondelli
      foaf_workInfoHomepage: http://www.librecat.org/personId=27EB676C-8706-11E9-9510-7717E6697425
    orcid: 0000-0002-3242-7020
  bibo_volume: 235
  dct_date: 2024^xs_gYear
  dct_language: eng
  dct_publisher: ML Research Press@
  dct_title: 'Compression of structured data with autoencoders: Provable benefit of
    nonlinearities and depth@'
...