conference paper published yes Matthew J Chalk author 2BAAC544-F248-11E8-B48F-1D18A9856A870000-0001-7782-4436 Olivier Marre author Gasper Tkacik author 3D494DCA-F248-11E8-B48F-1D18A9856A870000-0002-6699-1455 GaTk department NIPS: Neural Information Processing Systems In many applications, it is desirable to extract only the relevant aspects of data. A principled way to do this is the information bottleneck (IB) method, where one seeks a code that maximises information about a relevance variable, Y, while constraining the information encoded about the original data, X. Unfortunately however, the IB method is computationally demanding when data are high-dimensional and/or non-gaussian. Here we propose an approximate variational scheme for maximising a lower bound on the IB objective, analogous to variational EM. Using this method, we derive an IB algorithm to recover features that are both relevant and sparse. Finally, we demonstrate how kernelised versions of the algorithm can be used to address a broad range of problems with non-linear relation between X and Y. Neural Information Processing Systems Foundation2016Barcelona, Spain eng 1605.07332 291965-1973 https://papers.nips.cc/paper/6101-relevant-sparse-codes-with-variational-information-bottleneck Chalk MJ, Marre O, Tkačik G. Relevant sparse codes with variational information bottleneck. In: Vol 29. Neural Information Processing Systems Foundation; 2016:1965-1973. Chalk, M. J., Marre, O., &#38; Tkačik, G. (2016). Relevant sparse codes with variational information bottleneck (Vol. 29, pp. 1965–1973). Presented at the NIPS: Neural Information Processing Systems, Barcelona, Spain: Neural Information Processing Systems Foundation. M.J. Chalk, O. Marre, G. Tkačik, in:, Neural Information Processing Systems Foundation, 2016, pp. 1965–1973. Chalk, Matthew J., et al. <i>Relevant Sparse Codes with Variational Information Bottleneck</i>. Vol. 29, Neural Information Processing Systems Foundation, 2016, pp. 1965–73. Chalk, Matthew J, Olivier Marre, and Gašper Tkačik. “Relevant Sparse Codes with Variational Information Bottleneck,” 29:1965–73. Neural Information Processing Systems Foundation, 2016. Chalk MJ, Marre O, Tkačik G. 2016. Relevant sparse codes with variational information bottleneck. NIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 29, 1965–1973. M. J. Chalk, O. Marre, and G. Tkačik, “Relevant sparse codes with variational information bottleneck,” presented at the NIPS: Neural Information Processing Systems, Barcelona, Spain, 2016, vol. 29, pp. 1965–1973. 10822018-12-11T11:50:03Z2025-06-03T11:33:51Z