---
_id: '1082'
abstract:
- lang: eng
  text: In many applications, it is desirable to extract only the relevant aspects
    of data. A principled way to do this is the information bottleneck (IB) method,
    where one seeks a code that maximises information about a relevance variable,
    Y, while constraining the information encoded about the original data, X. Unfortunately
    however, the IB method is computationally demanding when data are high-dimensional
    and/or non-gaussian. Here we propose an approximate variational scheme for maximising
    a lower bound on the IB objective, analogous to variational EM. Using this method,
    we derive an IB algorithm to recover features that are both relevant and sparse.
    Finally, we demonstrate how kernelised versions of the algorithm can be used to
    address a broad range of problems with non-linear relation between X and Y.
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Matthew J
  full_name: Chalk, Matthew J
  id: 2BAAC544-F248-11E8-B48F-1D18A9856A87
  last_name: Chalk
  orcid: 0000-0001-7782-4436
- first_name: Olivier
  full_name: Marre, Olivier
  last_name: Marre
- first_name: Gasper
  full_name: Tkacik, Gasper
  id: 3D494DCA-F248-11E8-B48F-1D18A9856A87
  last_name: Tkacik
  orcid: 0000-0002-6699-1455
citation:
  ama: 'Chalk MJ, Marre O, Tkačik G. Relevant sparse codes with variational information
    bottleneck. In: Vol 29. Neural Information Processing Systems Foundation; 2016:1965-1973.'
  apa: 'Chalk, M. J., Marre, O., & Tkačik, G. (2016). Relevant sparse codes with
    variational information bottleneck (Vol. 29, pp. 1965–1973). Presented at the
    NIPS: Neural Information Processing Systems, Barcelona, Spain: Neural Information
    Processing Systems Foundation.'
  chicago: Chalk, Matthew J, Olivier Marre, and Gašper Tkačik. “Relevant Sparse Codes
    with Variational Information Bottleneck,” 29:1965–73. Neural Information Processing
    Systems Foundation, 2016.
  ieee: 'M. J. Chalk, O. Marre, and G. Tkačik, “Relevant sparse codes with variational
    information bottleneck,” presented at the NIPS: Neural Information Processing
    Systems, Barcelona, Spain, 2016, vol. 29, pp. 1965–1973.'
  ista: 'Chalk MJ, Marre O, Tkačik G. 2016. Relevant sparse codes with variational
    information bottleneck. NIPS: Neural Information Processing Systems, Advances
    in Neural Information Processing Systems, vol. 29, 1965–1973.'
  mla: Chalk, Matthew J., et al. <i>Relevant Sparse Codes with Variational Information
    Bottleneck</i>. Vol. 29, Neural Information Processing Systems Foundation, 2016,
    pp. 1965–73.
  short: M.J. Chalk, O. Marre, G. Tkačik, in:, Neural Information Processing Systems
    Foundation, 2016, pp. 1965–1973.
conference:
  end_date: 2016-12-10
  location: Barcelona, Spain
  name: 'NIPS: Neural Information Processing Systems'
  start_date: 2016-12-05
date_created: 2018-12-11T11:50:03Z
date_published: 2016-12-01T00:00:00Z
date_updated: 2025-06-03T11:33:51Z
day: '01'
department:
- _id: GaTk
external_id:
  arxiv:
  - '1605.07332'
intvolume: '        29'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://arxiv.org/abs/1605.07332
month: '12'
oa: 1
oa_version: Preprint
page: 1965-1973
publication_status: published
publisher: Neural Information Processing Systems Foundation
publist_id: '6298'
quality_controlled: '1'
related_material:
  link:
  - relation: other
    url: https://papers.nips.cc/paper/6101-relevant-sparse-codes-with-variational-information-bottleneck
scopus_import: '1'
status: public
title: Relevant sparse codes with variational information bottleneck
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 29
year: '2016'
...