--- _id: '14171' abstract: - lang: eng text: "This paper demonstrates how to recover causal graphs from the score of the\r\ndata distribution in non-linear additive (Gaussian) noise models. Using score\r\nmatching algorithms as a building block, we show how to design a new generation\r\nof scalable causal discovery methods. To showcase our approach, we also propose\r\na new efficient method for approximating the score's Jacobian, enabling to\r\nrecover the causal graph. Empirically, we find that the new algorithm, called\r\nSCORE, is competitive with state-of-the-art causal discovery methods while\r\nbeing significantly faster." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Paul full_name: Rolland, Paul last_name: Rolland - first_name: Volkan full_name: Cevher, Volkan last_name: Cevher - first_name: Matthäus full_name: Kleindessner, Matthäus last_name: Kleindessner - first_name: Chris full_name: Russel, Chris last_name: Russel - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Dominik full_name: Janzing, Dominik last_name: Janzing - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: 'Rolland P, Cevher V, Kleindessner M, et al. Score matching enables causal discovery of nonlinear additive noise  models. In: Proceedings of the 39th International Conference on Machine Learning. Vol 162. ML Research Press; 2022:18741-18753.' apa: 'Rolland, P., Cevher, V., Kleindessner, M., Russel, C., Schölkopf, B., Janzing, D., & Locatello, F. (2022). Score matching enables causal discovery of nonlinear additive noise  models. In Proceedings of the 39th International Conference on Machine Learning (Vol. 162, pp. 18741–18753). Baltimore, MD, United States: ML Research Press.' chicago: Rolland, Paul, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, and Francesco Locatello. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise  Models.” In Proceedings of the 39th International Conference on Machine Learning, 162:18741–53. ML Research Press, 2022. ieee: P. Rolland et al., “Score matching enables causal discovery of nonlinear additive noise  models,” in Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, United States, 2022, vol. 162, pp. 18741–18753. ista: Rolland P, Cevher V, Kleindessner M, Russel C, Schölkopf B, Janzing D, Locatello F. 2022. Score matching enables causal discovery of nonlinear additive noise  models. Proceedings of the 39th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 162, 18741–18753. mla: Rolland, Paul, et al. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise  Models.” Proceedings of the 39th International Conference on Machine Learning, vol. 162, ML Research Press, 2022, pp. 18741–53. short: P. Rolland, V. Cevher, M. Kleindessner, C. Russel, B. Schölkopf, D. Janzing, F. Locatello, in:, Proceedings of the 39th International Conference on Machine Learning, ML Research Press, 2022, pp. 18741–18753. conference: end_date: 2022-07-23 location: Baltimore, MD, United States name: International Conference on Machine Learning start_date: 2022-07-17 date_created: 2023-08-22T14:00:18Z date_published: 2022-07-22T00:00:00Z date_updated: 2023-09-11T10:14:20Z day: '22' department: - _id: FrLo extern: '1' external_id: arxiv: - '2203.04413' intvolume: ' 162' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2203.04413 month: '07' oa: 1 oa_version: Preprint page: 18741-18753 publication: Proceedings of the 39th International Conference on Machine Learning publication_status: published publisher: ML Research Press quality_controlled: '1' status: public title: Score matching enables causal discovery of nonlinear additive noise models type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 162 year: '2022' ... --- _id: '14172' abstract: - lang: eng text: "An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world. In this paper, we test whether 17 unsupervised, weakly supervised, and fully supervised representation learning approaches correctly infer the generative factors of variation in simple datasets (dSprites, Shapes3D, MPI3D) from controlled environments, and on our contributed CelebGlow dataset. In contrast to prior robustness work that introduces novel factors of variation during test time, such as blur or other (un)structured noise, we here recompose, interpolate, or extrapolate only existing factors of variation from the training data set (e.g., small and medium-sized objects during training and large objects during testing). Models\r\nthat learn the correct mechanism should be able to generalize to this benchmark. In total, we train and test 2000+ models and observe that all of them struggle to learn the underlying mechanism regardless of supervision signal and architectural bias. Moreover, the generalization capabilities of all tested models drop significantly as we move from artificial datasets towards\r\nmore realistic real-world datasets. Despite their inability to identify the correct mechanism, the models are quite modular as their ability to infer other in-distribution factors remains fairly stable, providing only a single factoris out-of-distribution. These results point to an important yet understudied problem of learning mechanistic models of observations that can facilitate\r\ngeneralization." article_processing_charge: No arxiv: 1 author: - first_name: Lukas full_name: Schott, Lukas last_name: Schott - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Peter full_name: Gehler, Peter last_name: Gehler - first_name: Chris full_name: Russell, Chris last_name: Russell - first_name: Matthias full_name: Bethge, Matthias last_name: Bethge - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Wieland full_name: Brendel, Wieland last_name: Brendel citation: ama: 'Schott L, Kügelgen J von, Träuble F, et al. Visual representation learning does not generalize strongly within the  same domain. In: 10th International Conference on Learning Representations. ; 2022.' apa: Schott, L., Kügelgen, J. von, Träuble, F., Gehler, P., Russell, C., Bethge, M., … Brendel, W. (2022). Visual representation learning does not generalize strongly within the  same domain. In 10th International Conference on Learning Representations. Virtual. chicago: Schott, Lukas, Julius von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bernhard Schölkopf, Francesco Locatello, and Wieland Brendel. “Visual Representation Learning Does Not Generalize Strongly within the  Same Domain.” In 10th International Conference on Learning Representations, 2022. ieee: L. Schott et al., “Visual representation learning does not generalize strongly within the  same domain,” in 10th International Conference on Learning Representations, Virtual, 2022. ista: 'Schott L, Kügelgen J von, Träuble F, Gehler P, Russell C, Bethge M, Schölkopf B, Locatello F, Brendel W. 2022. Visual representation learning does not generalize strongly within the  same domain. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Schott, Lukas, et al. “Visual Representation Learning Does Not Generalize Strongly within the  Same Domain.” 10th International Conference on Learning Representations, 2022. short: L. Schott, J. von Kügelgen, F. Träuble, P. Gehler, C. Russell, M. Bethge, B. Schölkopf, F. Locatello, W. Brendel, in:, 10th International Conference on Learning Representations, 2022. conference: end_date: 2022-04-29 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2022-04-25 date_created: 2023-08-22T14:00:50Z date_published: 2022-04-25T00:00:00Z date_updated: 2023-09-11T09:40:52Z day: '25' department: - _id: FrLo extern: '1' external_id: arxiv: - '2107.08221' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2107.08221 month: '04' oa: 1 oa_version: Preprint publication: 10th International Conference on Learning Representations publication_status: published quality_controlled: '1' status: public title: Visual representation learning does not generalize strongly within the same domain type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ... --- _id: '14173' abstract: - lang: eng text: "Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e.g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations. While sharing the same aspirational goal, these approaches have never been tested under the same\r\nexperimental conditions on real data. In this paper, we take a unified view of previous work, highlighting message discrepancies that we address empirically, and providing recommendations on how to measure the robustness of a model and how to improve it. To this end, we collect 172 publicly available dataset pairs for training and out-of-distribution evaluation of accuracy, calibration error, adversarial attacks, environment invariance, and synthetic corruptions. We fine-tune over 31k networks, from nine different architectures in the many- and\r\nfew-shot setting. Our findings confirm that in- and out-of-distribution accuracies tend to increase jointly, but show that their relation is largely dataset-dependent, and in general more nuanced and more complex than posited by previous, smaller scale studies." alternative_title: - Advances in Neural Information Processing Systems article_processing_charge: No arxiv: 1 author: - first_name: Florian full_name: Wenzel, Florian last_name: Wenzel - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Peter Vincent full_name: Gehler, Peter Vincent last_name: Gehler - first_name: Carl-Johann Simon-Gabriel full_name: Carl-Johann Simon-Gabriel, Carl-Johann Simon-Gabriel last_name: Carl-Johann Simon-Gabriel - first_name: Max full_name: Horn, Max last_name: Horn - first_name: Dominik full_name: Zietlow, Dominik last_name: Zietlow - first_name: David full_name: Kernert, David last_name: Kernert - first_name: Chris full_name: Russell, Chris last_name: Russell - first_name: Thomas full_name: Brox, Thomas last_name: Brox - first_name: Bernt full_name: Schiele, Bernt last_name: Schiele - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: 'Wenzel F, Dittadi A, Gehler PV, et al. Assaying out-of-distribution generalization in transfer learning. In: 36th Conference on Neural Information Processing Systems. Vol 35. Neural Information Processing Systems Foundation; 2022:7181-7198.' apa: 'Wenzel, F., Dittadi, A., Gehler, P. V., Carl-Johann Simon-Gabriel, C.-J. S.-G., Horn, M., Zietlow, D., … Locatello, F. (2022). Assaying out-of-distribution generalization in transfer learning. In 36th Conference on Neural Information Processing Systems (Vol. 35, pp. 7181–7198). New Orleans, LA, United States: Neural Information Processing Systems Foundation.' chicago: Wenzel, Florian, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, et al. “Assaying Out-of-Distribution Generalization in Transfer Learning.” In 36th Conference on Neural Information Processing Systems, 35:7181–98. Neural Information Processing Systems Foundation, 2022. ieee: F. Wenzel et al., “Assaying out-of-distribution generalization in transfer learning,” in 36th Conference on Neural Information Processing Systems, New Orleans, LA, United States, 2022, vol. 35, pp. 7181–7198. ista: 'Wenzel F, Dittadi A, Gehler PV, Carl-Johann Simon-Gabriel C-JS-G, Horn M, Zietlow D, Kernert D, Russell C, Brox T, Schiele B, Schölkopf B, Locatello F. 2022. Assaying out-of-distribution generalization in transfer learning. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 35, 7181–7198.' mla: Wenzel, Florian, et al. “Assaying Out-of-Distribution Generalization in Transfer Learning.” 36th Conference on Neural Information Processing Systems, vol. 35, Neural Information Processing Systems Foundation, 2022, pp. 7181–98. short: F. Wenzel, A. Dittadi, P.V. Gehler, C.-J.S.-G. Carl-Johann Simon-Gabriel, M. Horn, D. Zietlow, D. Kernert, C. Russell, T. Brox, B. Schiele, B. Schölkopf, F. Locatello, in:, 36th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2022, pp. 7181–7198. conference: end_date: 2022-12-09 location: New Orleans, LA, United States name: 'NeurIPS: Neural Information Processing Systems' start_date: 2022-11-28 date_created: 2023-08-22T14:01:13Z date_published: 2022-12-15T00:00:00Z date_updated: 2023-09-06T10:34:43Z day: '15' department: - _id: FrLo extern: '1' external_id: arxiv: - '2207.09239' intvolume: ' 35' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2207.09239 month: '12' oa: 1 oa_version: Preprint page: 7181-7198 publication: 36th Conference on Neural Information Processing Systems publication_identifier: isbn: - '9781713871088' publication_status: published publisher: Neural Information Processing Systems Foundation quality_controlled: '1' scopus_import: '1' status: public title: Assaying out-of-distribution generalization in transfer learning type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 35 year: '2022' ... --- _id: '14174' abstract: - lang: eng text: "Building sample-efficient agents that generalize out-of-distribution (OOD) in real-world settings remains a fundamental unsolved problem on the path towards achieving higher-level cognition. One particularly promising approach is to begin with low-dimensional, pretrained representations of our world, which should facilitate efficient downstream learning and generalization. By training 240 representations and over 10,000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of\r\npretrained VAE-based representations affect the OOD generalization of downstream agents. We observe that many agents are surprisingly robust to realistic distribution shifts, including the challenging sim-to-real case. In addition, we find that the generalization performance of a simple downstream proxy task reliably predicts the generalization performance of our RL agents\r\nunder a wide range of OOD settings. Such proxy tasks can thus be used to select pretrained representations that will lead to agents that generalize." article_processing_charge: No arxiv: 1 author: - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Manuel full_name: Wüthrich, Manuel last_name: Wüthrich - first_name: Felix full_name: Widmaier, Felix last_name: Widmaier - first_name: Peter full_name: Gehler, Peter last_name: Gehler - first_name: Ole full_name: Winther, Ole last_name: Winther - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer citation: ama: 'Dittadi A, Träuble F, Wüthrich M, et al. The role of pretrained representations for the OOD generalization of  reinforcement learning agents. In: 10th International Conference on Learning Representations. ; 2022.' apa: Dittadi, A., Träuble, F., Wüthrich, M., Widmaier, F., Gehler, P., Winther, O., … Bauer, S. (2022). The role of pretrained representations for the OOD generalization of  reinforcement learning agents. In 10th International Conference on Learning Representations. Virtual. chicago: Dittadi, Andrea, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. “The Role of Pretrained Representations for the OOD Generalization of  Reinforcement Learning Agents.” In 10th International Conference on Learning Representations, 2022. ieee: A. Dittadi et al., “The role of pretrained representations for the OOD generalization of  reinforcement learning agents,” in 10th International Conference on Learning Representations, Virtual, 2022. ista: 'Dittadi A, Träuble F, Wüthrich M, Widmaier F, Gehler P, Winther O, Locatello F, Bachem O, Schölkopf B, Bauer S. 2022. The role of pretrained representations for the OOD generalization of  reinforcement learning agents. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Dittadi, Andrea, et al. “The Role of Pretrained Representations for the OOD Generalization of  Reinforcement Learning Agents.” 10th International Conference on Learning Representations, 2022. short: A. Dittadi, F. Träuble, M. Wüthrich, F. Widmaier, P. Gehler, O. Winther, F. Locatello, O. Bachem, B. Schölkopf, S. Bauer, in:, 10th International Conference on Learning Representations, 2022. conference: end_date: 2022-04-29 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2022-04-25 date_created: 2023-08-22T14:02:13Z date_published: 2022-04-25T00:00:00Z date_updated: 2023-09-11T09:48:36Z day: '25' department: - _id: FrLo extern: '1' external_id: arxiv: - '2107.05686' language: - iso: eng main_file_link: - open_access: '1' url: ' https://doi.org/10.48550/arXiv.2107.05686' month: '04' oa: 1 oa_version: Preprint publication: 10th International Conference on Learning Representations publication_status: published quality_controlled: '1' status: public title: The role of pretrained representations for the OOD generalization of reinforcement learning agents type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ... --- _id: '14175' abstract: - lang: eng text: "Predicting the future trajectory of a moving agent can be easy when the past trajectory continues smoothly but is challenging when complex interactions with other agents are involved. Recent deep learning approaches for trajectory prediction show promising performance and partially attribute this to successful reasoning about agent-agent interactions. However, it remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions\r\nof different cues to model performance based on a variant of Shapley values. Applying this procedure to state-of-the-art trajectory prediction methods on standard benchmark datasets shows that they are, in fact, unable to reason about interactions. Instead, the past trajectory of the target is the only feature used for predicting its future. For a task with richer social\r\ninteraction patterns, on the other hand, the tested models do pick up such interactions to a certain extent, as quantified by our feature attribution method. We discuss the limits of the proposed method and its links to causality." article_processing_charge: No arxiv: 1 author: - first_name: Osama full_name: Makansi, Osama last_name: Makansi - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Peter full_name: Gehler, Peter last_name: Gehler - first_name: Dominik full_name: Janzing, Dominik last_name: Janzing - first_name: Thomas full_name: Brox, Thomas last_name: Brox - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Makansi O, Kügelgen J von, Locatello F, et al. You mostly walk alone: Analyzing feature attribution in trajectory prediction. In: 10th International Conference on Learning Representations. ; 2022.' apa: 'Makansi, O., Kügelgen, J. von, Locatello, F., Gehler, P., Janzing, D., Brox, T., & Schölkopf, B. (2022). You mostly walk alone: Analyzing feature attribution in trajectory prediction. In 10th International Conference on Learning Representations. Virtual.' chicago: 'Makansi, Osama, Julius von Kügelgen, Francesco Locatello, Peter Gehler, Dominik Janzing, Thomas Brox, and Bernhard Schölkopf. “You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction.” In 10th International Conference on Learning Representations, 2022.' ieee: 'O. Makansi et al., “You mostly walk alone: Analyzing feature attribution in trajectory prediction,” in 10th International Conference on Learning Representations, Virtual, 2022.' ista: 'Makansi O, Kügelgen J von, Locatello F, Gehler P, Janzing D, Brox T, Schölkopf B. 2022. You mostly walk alone: Analyzing feature attribution in trajectory prediction. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: 'Makansi, Osama, et al. “You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction.” 10th International Conference on Learning Representations, 2022.' short: O. Makansi, J. von Kügelgen, F. Locatello, P. Gehler, D. Janzing, T. Brox, B. Schölkopf, in:, 10th International Conference on Learning Representations, 2022. conference: end_date: 2022-04-29 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2022-04-25 date_created: 2023-08-22T14:02:34Z date_published: 2022-04-25T00:00:00Z date_updated: 2023-09-11T09:52:20Z day: '25' department: - _id: FrLo extern: '1' external_id: arxiv: - '2110.05304' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2110.05304 month: '04' oa: 1 oa_version: Preprint publication: 10th International Conference on Learning Representations publication_status: published quality_controlled: '1' status: public title: 'You mostly walk alone: Analyzing feature attribution in trajectory prediction' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ... --- _id: '14215' abstract: - lang: eng text: Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications. However, collaboration between these actors is difficult due to the heterogeneous nature of geospatial data modalities (e.g., multi-spectral images of various resolutions, timeseries, weather data) and diversity of tasks (e.g., regression of human activity indicators or detecting forest fires). In this work, we present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias, pre-trained on large amounts of unlabelled earth observation data in a self-supervised manner. We envision how such a model may facilitate cooperation between members of the community. We show preliminary results on the first step of the roadmap, where we instantiate an architecture that can process a wide variety of geospatial data modalities and demonstrate that it can achieve competitive performance with domain-specific architectures on tasks relating to the U.N.'s Sustainable Development Goals. article_processing_charge: No arxiv: 1 author: - first_name: Nasim full_name: Rahaman, Nasim last_name: Rahaman - first_name: Martin full_name: Weiss, Martin last_name: Weiss - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Alexandre full_name: Lacoste, Alexandre last_name: Lacoste - first_name: Yoshua full_name: Bengio, Yoshua last_name: Bengio - first_name: Chris full_name: Pal, Chris last_name: Pal - first_name: Li Erran full_name: Li, Li Erran last_name: Li - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Rahaman N, Weiss M, Träuble F, et al. A general purpose neural architecture for geospatial systems. In: 36th Conference on Neural Information Processing Systems.' apa: Rahaman, N., Weiss, M., Träuble, F., Locatello, F., Lacoste, A., Bengio, Y., … Schölkopf, B. (n.d.). A general purpose neural architecture for geospatial systems. In 36th Conference on Neural Information Processing Systems. New Orleans, LA, United States. chicago: Rahaman, Nasim, Martin Weiss, Frederik Träuble, Francesco Locatello, Alexandre Lacoste, Yoshua Bengio, Chris Pal, Li Erran Li, and Bernhard Schölkopf. “A General Purpose Neural Architecture for Geospatial Systems.” In 36th Conference on Neural Information Processing Systems, n.d. ieee: N. Rahaman et al., “A general purpose neural architecture for geospatial systems,” in 36th Conference on Neural Information Processing Systems, New Orleans, LA, United States. ista: 'Rahaman N, Weiss M, Träuble F, Locatello F, Lacoste A, Bengio Y, Pal C, Li LE, Schölkopf B. A general purpose neural architecture for geospatial systems. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems.' mla: Rahaman, Nasim, et al. “A General Purpose Neural Architecture for Geospatial Systems.” 36th Conference on Neural Information Processing Systems. short: N. Rahaman, M. Weiss, F. Träuble, F. Locatello, A. Lacoste, Y. Bengio, C. Pal, L.E. Li, B. Schölkopf, in:, 36th Conference on Neural Information Processing Systems, n.d. conference: end_date: 2022-12-09 location: New Orleans, LA, United States name: 'NeurIPS: Neural Information Processing Systems' start_date: 2022-11-28 date_created: 2023-08-22T14:21:47Z date_published: 2022-11-04T00:00:00Z date_updated: 2023-09-13T09:35:59Z day: '04' department: - _id: FrLo extern: '1' external_id: arxiv: - '2211.02348' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2211.02348 month: '11' oa: 1 oa_version: Preprint publication: 36th Conference on Neural Information Processing Systems publication_status: submitted quality_controlled: '1' status: public title: A general purpose neural architecture for geospatial systems type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ... --- _id: '14220' abstract: - lang: eng text: Although reinforcement learning has seen remarkable progress over the last years, solving robust dexterous object-manipulation tasks in multi-object settings remains a challenge. In this paper, we focus on models that can learn manipulation tasks in fixed multi-object settings and extrapolate this skill zero-shot without any drop in performance when the number of objects changes. We consider the generic task of bringing a specific cube out of a set to a goal position. We find that previous approaches, which primarily leverage attention and graph neural network-based architectures, do not generalize their skills when the number of input objects changes while scaling as K2. We propose an alternative plug-and-play module based on relational inductive biases to overcome these limitations. Besides exceeding performances in their training environment, we show that our approach, which scales linearly in K, allows agents to extrapolate and generalize zero-shot to any new object number. article_number: '2201.13388' article_processing_charge: No arxiv: 1 author: - first_name: Davide full_name: Mambelli, Davide last_name: Mambelli - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: Mambelli D, Träuble F, Bauer S, Schölkopf B, Locatello F. Compositional multi-object reinforcement learning with linear relation networks. arXiv. doi:10.48550/arXiv.2201.13388 apa: Mambelli, D., Träuble, F., Bauer, S., Schölkopf, B., & Locatello, F. (n.d.). Compositional multi-object reinforcement learning with linear relation networks. arXiv. https://doi.org/10.48550/arXiv.2201.13388 chicago: Mambelli, Davide, Frederik Träuble, Stefan Bauer, Bernhard Schölkopf, and Francesco Locatello. “Compositional Multi-Object Reinforcement Learning with Linear Relation Networks.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2201.13388. ieee: D. Mambelli, F. Träuble, S. Bauer, B. Schölkopf, and F. Locatello, “Compositional multi-object reinforcement learning with linear relation networks,” arXiv. . ista: Mambelli D, Träuble F, Bauer S, Schölkopf B, Locatello F. Compositional multi-object reinforcement learning with linear relation networks. arXiv, 2201.13388. mla: Mambelli, Davide, et al. “Compositional Multi-Object Reinforcement Learning with Linear Relation Networks.” ArXiv, 2201.13388, doi:10.48550/arXiv.2201.13388. short: D. Mambelli, F. Träuble, S. Bauer, B. Schölkopf, F. Locatello, ArXiv (n.d.). date_created: 2023-08-22T14:23:16Z date_published: 2022-01-31T00:00:00Z date_updated: 2024-10-14T12:27:39Z day: '31' department: - _id: FrLo doi: 10.48550/arXiv.2201.13388 extern: '1' external_id: arxiv: - '2201.13388' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2201.13388 month: '01' oa: 1 oa_version: Preprint publication: arXiv publication_status: submitted status: public title: Compositional multi-object reinforcement learning with linear relation networks type: preprint user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ... --- _id: '14117' abstract: - lang: eng text: 'The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.' article_processing_charge: No article_type: original arxiv: 1 author: - first_name: Bernhard full_name: Scholkopf, Bernhard last_name: Scholkopf - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Nan Rosemary full_name: Ke, Nan Rosemary last_name: Ke - first_name: Nal full_name: Kalchbrenner, Nal last_name: Kalchbrenner - first_name: Anirudh full_name: Goyal, Anirudh last_name: Goyal - first_name: Yoshua full_name: Bengio, Yoshua last_name: Bengio citation: ama: Scholkopf B, Locatello F, Bauer S, et al. Toward causal representation learning. Proceedings of the IEEE. 2021;109(5):612-634. doi:10.1109/jproc.2021.3058954 apa: Scholkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., & Bengio, Y. (2021). Toward causal representation learning. Proceedings of the IEEE. Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/jproc.2021.3058954 chicago: Scholkopf, Bernhard, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. “Toward Causal Representation Learning.” Proceedings of the IEEE. Institute of Electrical and Electronics Engineers, 2021. https://doi.org/10.1109/jproc.2021.3058954. ieee: B. Scholkopf et al., “Toward causal representation learning,” Proceedings of the IEEE, vol. 109, no. 5. Institute of Electrical and Electronics Engineers, pp. 612–634, 2021. ista: Scholkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, Bengio Y. 2021. Toward causal representation learning. Proceedings of the IEEE. 109(5), 612–634. mla: Scholkopf, Bernhard, et al. “Toward Causal Representation Learning.” Proceedings of the IEEE, vol. 109, no. 5, Institute of Electrical and Electronics Engineers, 2021, pp. 612–34, doi:10.1109/jproc.2021.3058954. short: B. Scholkopf, F. Locatello, S. Bauer, N.R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Proceedings of the IEEE 109 (2021) 612–634. date_created: 2023-08-21T12:19:30Z date_published: 2021-05-01T00:00:00Z date_updated: 2023-09-11T11:43:35Z day: '01' department: - _id: FrLo doi: 10.1109/jproc.2021.3058954 extern: '1' external_id: arxiv: - '2102.11107' intvolume: ' 109' issue: '5' keyword: - Electrical and Electronic Engineering language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.1109/JPROC.2021.3058954 month: '05' oa: 1 oa_version: Published Version page: 612-634 publication: Proceedings of the IEEE publication_identifier: eissn: - 1558-2256 issn: - 0018-9219 publication_status: published publisher: Institute of Electrical and Electronics Engineers quality_controlled: '1' scopus_import: '1' status: public title: Toward causal representation learning type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 109 year: '2021' ... --- _id: '14176' abstract: - lang: eng text: "Intensive care units (ICU) are increasingly looking towards machine learning for methods to provide online monitoring of critically ill patients. In machine learning, online monitoring is often formulated as a supervised learning problem. Recently, contrastive learning approaches have demonstrated promising improvements over competitive supervised benchmarks. These methods rely on well-understood data augmentation techniques developed for image data which do not apply to online monitoring. In this work, we overcome this limitation by\r\nsupplementing time-series data augmentation techniques with a novel contrastive\r\nlearning objective which we call neighborhood contrastive learning (NCL). Our objective explicitly groups together contiguous time segments from each patient while maintaining state-specific information. Our experiments demonstrate a marked improvement over existing work applying contrastive methods to medical time-series." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Hugo full_name: Yèche, Hugo last_name: Yèche - first_name: Gideon full_name: Dresdner, Gideon last_name: Dresdner - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Matthias full_name: Hüser, Matthias last_name: Hüser - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch citation: ama: 'Yèche H, Dresdner G, Locatello F, Hüser M, Rätsch G. Neighborhood contrastive learning applied to online patient monitoring. In: Proceedings of 38th International Conference on Machine Learning. Vol 139. ML Research Press; 2021:11964-11974.' apa: 'Yèche, H., Dresdner, G., Locatello, F., Hüser, M., & Rätsch, G. (2021). Neighborhood contrastive learning applied to online patient monitoring. In Proceedings of 38th International Conference on Machine Learning (Vol. 139, pp. 11964–11974). Virtual: ML Research Press.' chicago: Yèche, Hugo, Gideon Dresdner, Francesco Locatello, Matthias Hüser, and Gunnar Rätsch. “Neighborhood Contrastive Learning Applied to Online Patient Monitoring.” In Proceedings of 38th International Conference on Machine Learning, 139:11964–74. ML Research Press, 2021. ieee: H. Yèche, G. Dresdner, F. Locatello, M. Hüser, and G. Rätsch, “Neighborhood contrastive learning applied to online patient monitoring,” in Proceedings of 38th International Conference on Machine Learning, Virtual, 2021, vol. 139, pp. 11964–11974. ista: Yèche H, Dresdner G, Locatello F, Hüser M, Rätsch G. 2021. Neighborhood contrastive learning applied to online patient monitoring. Proceedings of 38th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 139, 11964–11974. mla: Yèche, Hugo, et al. “Neighborhood Contrastive Learning Applied to Online Patient Monitoring.” Proceedings of 38th International Conference on Machine Learning, vol. 139, ML Research Press, 2021, pp. 11964–74. short: H. Yèche, G. Dresdner, F. Locatello, M. Hüser, G. Rätsch, in:, Proceedings of 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 11964–11974. conference: end_date: 2021-07-24 location: Virtual name: International Conference on Machine Learning start_date: 2021-07-18 date_created: 2023-08-22T14:03:04Z date_published: 2021-08-01T00:00:00Z date_updated: 2023-09-11T10:16:55Z day: '01' department: - _id: FrLo extern: '1' external_id: arxiv: - '2106.05142' intvolume: ' 139' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2106.05142 month: '08' oa: 1 oa_version: Preprint page: 11964-11974 publication: Proceedings of 38th International Conference on Machine Learning publication_status: published publisher: ML Research Press quality_controlled: '1' scopus_import: '1' status: public title: Neighborhood contrastive learning applied to online patient monitoring type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 139 year: '2021' ... --- _id: '14177' abstract: - lang: eng text: "The focus of disentanglement approaches has been on identifying independent factors of variation in data. However, the causal variables underlying real-world observations are often not statistically independent. In this work, we bridge the gap to real-world scenarios by analyzing the behavior of the most prominent disentanglement approaches on correlated data in a large-scale empirical study (including 4260 models). We show and quantify that systematically induced correlations in the dataset are being learned and reflected in the latent representations, which has implications for downstream applications of disentanglement such as fairness. We also demonstrate how to resolve these latent correlations, either using weak supervision during\r\ntraining or by post-hoc correcting a pre-trained model with a small number of labels." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Elliot full_name: Creager, Elliot last_name: Creager - first_name: Niki full_name: Kilbertus, Niki last_name: Kilbertus - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Anirudh full_name: Goyal, Anirudh last_name: Goyal - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer citation: ama: 'Träuble F, Creager E, Kilbertus N, et al. On disentangled representations learned from correlated data. In: Proceedings of the 38th International Conference on Machine Learning. Vol 139. ML Research Press; 2021:10401-10412.' apa: 'Träuble, F., Creager, E., Kilbertus, N., Locatello, F., Dittadi, A., Goyal, A., … Bauer, S. (2021). On disentangled representations learned from correlated data. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 10401–10412). Virtual: ML Research Press.' chicago: Träuble, Frederik, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Schölkopf, and Stefan Bauer. “On Disentangled Representations Learned from Correlated Data.” In Proceedings of the 38th International Conference on Machine Learning, 139:10401–12. ML Research Press, 2021. ieee: F. Träuble et al., “On disentangled representations learned from correlated data,” in Proceedings of the 38th International Conference on Machine Learning, Virtual, 2021, vol. 139, pp. 10401–10412. ista: 'Träuble F, Creager E, Kilbertus N, Locatello F, Dittadi A, Goyal A, Schölkopf B, Bauer S. 2021. On disentangled representations learned from correlated data. Proceedings of the 38th International Conference on Machine Learning. ICML: International Conference on Machine Learning, PMLR, vol. 139, 10401–10412.' mla: Träuble, Frederik, et al. “On Disentangled Representations Learned from Correlated Data.” Proceedings of the 38th International Conference on Machine Learning, vol. 139, ML Research Press, 2021, pp. 10401–12. short: F. Träuble, E. Creager, N. Kilbertus, F. Locatello, A. Dittadi, A. Goyal, B. Schölkopf, S. Bauer, in:, Proceedings of the 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 10401–10412. conference: end_date: 2021-07-24 location: Virtual name: 'ICML: International Conference on Machine Learning' start_date: 2021-07-18 date_created: 2023-08-22T14:03:47Z date_published: 2021-08-01T00:00:00Z date_updated: 2023-09-11T10:18:48Z day: '01' department: - _id: FrLo extern: '1' external_id: arxiv: - '2006.07886' intvolume: ' 139' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2006.07886 month: '08' oa: 1 oa_version: Published Version page: 10401-10412 publication: Proceedings of the 38th International Conference on Machine Learning publication_status: published publisher: ML Research Press quality_controlled: '1' scopus_import: '1' status: public title: On disentangled representations learned from correlated data type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 139 year: '2021' ... --- _id: '14178' abstract: - lang: eng text: Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning. While disentangled representations were found to be useful for diverse tasks such as abstract reasoning and fair classification, their scalability and real-world impact remain questionable. We introduce a new high-resolution dataset with 1M simulated images and over 1,800 annotated real-world images of the same setup. In contrast to previous work, this new dataset exhibits correlations, a complex underlying structure, and allows to evaluate transfer to unseen simulated and real-world settings where the encoder i) remains in distribution or ii) is out of distribution. We propose new architectures in order to scale disentangled representation learning to realistic high-resolution settings and conduct a large-scale empirical study of disentangled representations on this dataset. We observe that disentanglement is a good predictor for out-of-distribution (OOD) task performance. article_processing_charge: No arxiv: 1 author: - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Manuel full_name: Wüthrich, Manuel last_name: Wüthrich - first_name: Vaibhav full_name: Agrawal, Vaibhav last_name: Agrawal - first_name: Ole full_name: Winther, Ole last_name: Winther - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Dittadi A, Träuble F, Locatello F, et al. On the transfer of disentangled representations in realistic settings. In: The Ninth International Conference on Learning Representations. ; 2021.' apa: Dittadi, A., Träuble, F., Locatello, F., Wüthrich, M., Agrawal, V., Winther, O., … Schölkopf, B. (2021). On the transfer of disentangled representations in realistic settings. In The Ninth International Conference on Learning Representations. Virtual. chicago: Dittadi, Andrea, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, and Bernhard Schölkopf. “On the Transfer of Disentangled Representations in Realistic Settings.” In The Ninth International Conference on Learning Representations, 2021. ieee: A. Dittadi et al., “On the transfer of disentangled representations in realistic settings,” in The Ninth International Conference on Learning Representations, Virtual, 2021. ista: 'Dittadi A, Träuble F, Locatello F, Wüthrich M, Agrawal V, Winther O, Bauer S, Schölkopf B. 2021. On the transfer of disentangled representations in realistic settings. The Ninth International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Dittadi, Andrea, et al. “On the Transfer of Disentangled Representations in Realistic Settings.” The Ninth International Conference on Learning Representations, 2021. short: A. Dittadi, F. Träuble, F. Locatello, M. Wüthrich, V. Agrawal, O. Winther, S. Bauer, B. Schölkopf, in:, The Ninth International Conference on Learning Representations, 2021. conference: end_date: 2021-05-07 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2021-05-03 date_created: 2023-08-22T14:04:16Z date_published: 2021-05-04T00:00:00Z date_updated: 2023-09-11T10:55:30Z day: '04' department: - _id: FrLo extern: '1' external_id: arxiv: - '2010.14407' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2010.14407 month: '05' oa: 1 oa_version: Preprint publication: The Ninth International Conference on Learning Representations publication_status: published quality_controlled: '1' status: public title: On the transfer of disentangled representations in realistic settings type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14179' abstract: - lang: eng text: Self-supervised representation learning has shown remarkable success in a number of domains. A common practice is to perform data augmentation via hand-crafted transformations intended to leave the semantics of the data invariant. We seek to understand the empirical success of this approach from a theoretical perspective. We formulate the augmentation process as a latent variable model by postulating a partition of the latent representation into a content component, which is assumed invariant to augmentation, and a style component, which is allowed to change. Unlike prior work on disentanglement and independent component analysis, we allow for both nontrivial statistical and causal dependencies in the latent space. We study the identifiability of the latent representation based on pairs of views of the observations and prove sufficient conditions that allow us to identify the invariant content partition up to an invertible mapping in both generative and discriminative settings. We find numerical simulations with dependent latent variables are consistent with our theory. Lastly, we introduce Causal3DIdent, a dataset of high-dimensional, visually complex images with rich causal dependencies, which we use to study the effect of data augmentations performed in practice. article_processing_charge: No arxiv: 1 author: - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Yash full_name: Sharma, Yash last_name: Sharma - first_name: Luigi full_name: Gresele, Luigi last_name: Gresele - first_name: Wieland full_name: Brendel, Wieland last_name: Brendel - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Michel full_name: Besserve, Michel last_name: Besserve - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: 'Kügelgen J von, Sharma Y, Gresele L, et al. Self-supervised learning with data augmentations provably isolates content from style. In: Advances in Neural Information Processing Systems. Vol 34. ; 2021:16451-16467.' apa: Kügelgen, J. von, Sharma, Y., Gresele, L., Brendel, W., Schölkopf, B., Besserve, M., & Locatello, F. (2021). Self-supervised learning with data augmentations provably isolates content from style. In Advances in Neural Information Processing Systems (Vol. 34, pp. 16451–16467). Virtual. chicago: Kügelgen, Julius von, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. “Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style.” In Advances in Neural Information Processing Systems, 34:16451–67, 2021. ieee: J. von Kügelgen et al., “Self-supervised learning with data augmentations provably isolates content from style,” in Advances in Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 16451–16467. ista: 'Kügelgen J von, Sharma Y, Gresele L, Brendel W, Schölkopf B, Besserve M, Locatello F. 2021. Self-supervised learning with data augmentations provably isolates content from style. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 16451–16467.' mla: Kügelgen, Julius von, et al. “Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style.” Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 16451–67. short: J. von Kügelgen, Y. Sharma, L. Gresele, W. Brendel, B. Schölkopf, M. Besserve, F. Locatello, in:, Advances in Neural Information Processing Systems, 2021, pp. 16451–16467. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:04:36Z date_published: 2021-06-08T00:00:00Z date_updated: 2023-09-11T10:33:19Z day: '08' department: - _id: FrLo extern: '1' external_id: arxiv: - '2106.04619' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2106.04619 month: '06' oa: 1 oa_version: Preprint page: 16451-16467 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: Self-supervised learning with data augmentations provably isolates content from style type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- _id: '14180' abstract: - lang: eng text: 'Modern neural network architectures can leverage large amounts of data to generalize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, a feat that is hypothesized to require compositional reasoning and reuse of knowledge. In this work, we present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which we call \emph{functions}. Inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate it in two distinct settings: image classification and visual abstract reasoning on Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-art in terms of systematic generalization. ' article_processing_charge: No arxiv: 1 author: - first_name: Nasim full_name: Rahaman, Nasim last_name: Rahaman - first_name: Muhammad Waleed full_name: Gondal, Muhammad Waleed last_name: Gondal - first_name: Shruti full_name: Joshi, Shruti last_name: Joshi - first_name: Peter full_name: Gehler, Peter last_name: Gehler - first_name: Yoshua full_name: Bengio, Yoshua last_name: Bengio - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Rahaman N, Gondal MW, Joshi S, et al. Dynamic inference with neural interpreters. In: Advances in Neural Information Processing Systems. Vol 34. ; 2021:10985-10998.' apa: Rahaman, N., Gondal, M. W., Joshi, S., Gehler, P., Bengio, Y., Locatello, F., & Schölkopf, B. (2021). Dynamic inference with neural interpreters. In Advances in Neural Information Processing Systems (Vol. 34, pp. 10985–10998). Virtual. chicago: Rahaman, Nasim, Muhammad Waleed Gondal, Shruti Joshi, Peter Gehler, Yoshua Bengio, Francesco Locatello, and Bernhard Schölkopf. “Dynamic Inference with Neural Interpreters.” In Advances in Neural Information Processing Systems, 34:10985–98, 2021. ieee: N. Rahaman et al., “Dynamic inference with neural interpreters,” in Advances in Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 10985–10998. ista: 'Rahaman N, Gondal MW, Joshi S, Gehler P, Bengio Y, Locatello F, Schölkopf B. 2021. Dynamic inference with neural interpreters. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 10985–10998.' mla: Rahaman, Nasim, et al. “Dynamic Inference with Neural Interpreters.” Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 10985–98. short: N. Rahaman, M.W. Gondal, S. Joshi, P. Gehler, Y. Bengio, F. Locatello, B. Schölkopf, in:, Advances in Neural Information Processing Systems, 2021, pp. 10985–10998. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:04:55Z date_published: 2021-10-12T00:00:00Z date_updated: 2024-10-14T12:27:25Z day: '12' department: - _id: FrLo extern: '1' external_id: arxiv: - '2110.06399' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2110.06399 month: '10' oa: 1 oa_version: Preprint page: 10985-10998 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: Dynamic inference with neural interpreters type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- _id: '14181' abstract: - lang: eng text: Variational Inference makes a trade-off between the capacity of the variational family and the tractability of finding an approximate posterior distribution. Instead, Boosting Variational Inference allows practitioners to obtain increasingly good posterior approximations by spending more compute. The main obstacle to widespread adoption of Boosting Variational Inference is the amount of resources necessary to improve over a strong Variational Inference baseline. In our work, we trace this limitation back to the global curvature of the KL-divergence. We characterize how the global curvature impacts time and memory consumption, address the problem with the notion of local curvature, and provide a novel approximate backtracking algorithm for estimating local curvature. We give new theoretical convergence rates for our algorithms and provide experimental validation on synthetic and real-world datasets. article_processing_charge: No arxiv: 1 author: - first_name: Gideon full_name: Dresdner, Gideon last_name: Dresdner - first_name: Saurav full_name: Shekhar, Saurav last_name: Shekhar - first_name: Fabian full_name: Pedregosa, Fabian last_name: Pedregosa - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch citation: ama: 'Dresdner G, Shekhar S, Pedregosa F, Locatello F, Rätsch G. Boosting variational inference with locally adaptive step-sizes. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence; 2021:2337-2343. doi:10.24963/ijcai.2021/322' apa: 'Dresdner, G., Shekhar, S., Pedregosa, F., Locatello, F., & Rätsch, G. (2021). Boosting variational inference with locally adaptive step-sizes. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (pp. 2337–2343). Montreal, Canada: International Joint Conferences on Artificial Intelligence. https://doi.org/10.24963/ijcai.2021/322' chicago: Dresdner, Gideon, Saurav Shekhar, Fabian Pedregosa, Francesco Locatello, and Gunnar Rätsch. “Boosting Variational Inference with Locally Adaptive Step-Sizes.” In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2337–43. International Joint Conferences on Artificial Intelligence, 2021. https://doi.org/10.24963/ijcai.2021/322. ieee: G. Dresdner, S. Shekhar, F. Pedregosa, F. Locatello, and G. Rätsch, “Boosting variational inference with locally adaptive step-sizes,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, Canada, 2021, pp. 2337–2343. ista: 'Dresdner G, Shekhar S, Pedregosa F, Locatello F, Rätsch G. 2021. Boosting variational inference with locally adaptive step-sizes. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. IJCAI: International Joint Conference on Artificial Intelligence, 2337–2343.' mla: Dresdner, Gideon, et al. “Boosting Variational Inference with Locally Adaptive Step-Sizes.” Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence, 2021, pp. 2337–43, doi:10.24963/ijcai.2021/322. short: G. Dresdner, S. Shekhar, F. Pedregosa, F. Locatello, G. Rätsch, in:, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence, 2021, pp. 2337–2343. conference: end_date: 2021-08-27 location: Montreal, Canada name: 'IJCAI: International Joint Conference on Artificial Intelligence' start_date: 2021-08-19 date_created: 2023-08-22T14:05:14Z date_published: 2021-05-19T00:00:00Z date_updated: 2023-09-11T11:14:30Z day: '19' department: - _id: FrLo doi: 10.24963/ijcai.2021/322 extern: '1' external_id: arxiv: - '2105.09240' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2105.09240 month: '05' oa: 1 oa_version: Published Version page: 2337-2343 publication: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence publication_identifier: eisbn: - '9780999241196' publication_status: published publisher: International Joint Conferences on Artificial Intelligence quality_controlled: '1' status: public title: Boosting variational inference with locally adaptive step-sizes type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14182' abstract: - lang: eng text: "When machine learning systems meet real world applications, accuracy is only\r\none of several requirements. In this paper, we assay a complementary\r\nperspective originating from the increasing availability of pre-trained and\r\nregularly improving state-of-the-art models. While new improved models develop\r\nat a fast pace, downstream tasks vary more slowly or stay constant. Assume that\r\nwe have a large unlabelled data set for which we want to maintain accurate\r\npredictions. Whenever a new and presumably better ML models becomes available,\r\nwe encounter two problems: (i) given a limited budget, which data points should\r\nbe re-evaluated using the new model?; and (ii) if the new predictions differ\r\nfrom the current ones, should we update? Problem (i) is about compute cost,\r\nwhich matters for very large data sets and models. Problem (ii) is about\r\nmaintaining consistency of the predictions, which can be highly relevant for\r\ndownstream applications; our demand is to avoid negative flips, i.e., changing\r\ncorrect to incorrect predictions. In this paper, we formalize the Prediction\r\nUpdate Problem and present an efficient probabilistic approach as answer to the\r\nabove questions. In extensive experiments on standard classification benchmark\r\ndata sets, we show that our method outperforms alternative strategies along key\r\nmetrics for backward-compatible prediction updates." article_processing_charge: No arxiv: 1 author: - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Matthäus full_name: Kleindessner, Matthäus last_name: Kleindessner - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Peter full_name: Gehler, Peter last_name: Gehler citation: ama: 'Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. Backward-compatible prediction updates: A probabilistic approach. In: 35th Conference on Neural Information Processing Systems. Vol 34. ; 2021:116-128.' apa: 'Träuble, F., Kügelgen, J. von, Kleindessner, M., Locatello, F., Schölkopf, B., & Gehler, P. (2021). Backward-compatible prediction updates: A probabilistic approach. In 35th Conference on Neural Information Processing Systems (Vol. 34, pp. 116–128). Virtual.' chicago: 'Träuble, Frederik, Julius von Kügelgen, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, and Peter Gehler. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” In 35th Conference on Neural Information Processing Systems, 34:116–28, 2021.' ieee: 'F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, and P. Gehler, “Backward-compatible prediction updates: A probabilistic approach,” in 35th Conference on Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 116–128.' ista: 'Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. 2021. Backward-compatible prediction updates: A probabilistic approach. 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 116–128.' mla: 'Träuble, Frederik, et al. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” 35th Conference on Neural Information Processing Systems, vol. 34, 2021, pp. 116–28.' short: F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, P. Gehler, in:, 35th Conference on Neural Information Processing Systems, 2021, pp. 116–128. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:05:41Z date_published: 2021-07-02T00:00:00Z date_updated: 2023-09-11T11:31:59Z day: '02' department: - _id: FrLo extern: '1' external_id: arxiv: - '2107.01057' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2107.01057 month: '07' oa: 1 oa_version: Preprint page: 116-128 publication: 35th Conference on Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: 'Backward-compatible prediction updates: A probabilistic approach' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- OA_place: repository _id: '14185' abstract: - lang: eng text: A method involves receiving a perceptual representation including a plurality of feature vectors, and initializing a plurality of slot vectors represented by a neural network memory unit. Each respective slot vector is configured to represent a corresponding entity in the perceptual representation. The method also involves determining an attention matrix based on a product of the plurality of feature vectors transformed by a key function and the plurality of slot vectors transformed by a query function. Each respective value of a plurality of values along each respective dimension of the attention matrix is normalized with respect to the plurality of values. The method additionally involves determining an update matrix based on the plurality of feature vectors transformed by a value function and the attention matrix, and updating the plurality of slot vectors based on the update matrix by way of the neural network memory unit. applicant: - Google LLC application_date: 2020-07-13 application_number: '16 / 927,018 ' article_processing_charge: No arxiv: 1 author: - first_name: Dirk full_name: Weissenborn, Dirk last_name: Weissenborn - first_name: Jakob full_name: Uszkoreit, Jakob last_name: Uszkoreit - first_name: Thomas full_name: Unterthiner, Thomas last_name: Unterthiner - first_name: Aravindh full_name: Mahendran, Aravindh last_name: Mahendran - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Thomas full_name: Kipf, Thomas last_name: Kipf - first_name: Georg full_name: Heigold, Georg last_name: Heigold - first_name: Alexey full_name: Dosovitskiy, Alexey last_name: Dosovitskiy citation: ama: Weissenborn D, Uszkoreit J, Unterthiner T, et al. Object-centric learning with slot attention. 2021. apa: Weissenborn, D., Uszkoreit, J., Unterthiner, T., Mahendran, A., Locatello, F., Kipf, T., … Dosovitskiy, A. (2021). Object-centric learning with slot attention. chicago: Weissenborn, Dirk, Jakob Uszkoreit, Thomas Unterthiner, Aravindh Mahendran, Francesco Locatello, Thomas Kipf, Georg Heigold, and Alexey Dosovitskiy. “Object-Centric Learning with Slot Attention,” 2021. ieee: D. Weissenborn et al., “Object-centric learning with slot attention.” 2021. ista: Weissenborn D, Uszkoreit J, Unterthiner T, Mahendran A, Locatello F, Kipf T, Heigold G, Dosovitskiy A. 2021. Object-centric learning with slot attention. mla: Weissenborn, Dirk, et al. Object-Centric Learning with Slot Attention. 2021. short: D. Weissenborn, J. Uszkoreit, T. Unterthiner, A. Mahendran, F. Locatello, T. Kipf, G. Heigold, A. Dosovitskiy, (2021). date_created: 2023-08-22T14:07:06Z date_published: 2021-12-09T00:00:00Z date_updated: 2025-01-31T11:35:46Z day: '09' department: - _id: FrLo extern: '1' external_id: arxiv: - '2006.15055' ipc: G06N 3/063 ; G06N 3/08 ; G06F 17/16 ipn: US20210383199A1 main_file_link: - open_access: '1' url: https://patents.google.com/patent/US20210383199A1/en month: '12' oa: 1 oa_version: Published Version publication_date: 2021-12-09 status: public title: Object-centric learning with slot attention type: patent user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2021' ... --- _id: '14221' abstract: - lang: eng text: 'The world is structured in countless ways. It may be prudent to enforce corresponding structural properties to a learning algorithm''s solution, such as incorporating prior beliefs, natural constraints, or causal structures. Doing so may translate to faster, more accurate, and more flexible models, which may directly relate to real-world impact. In this dissertation, we consider two different research areas that concern structuring a learning algorithm''s solution: when the structure is known and when it has to be discovered.' article_number: '2111.13693' article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: Locatello F. Enforcing and discovering structure in machine learning. arXiv. doi:10.48550/arXiv.2111.13693 apa: Locatello, F. (n.d.). Enforcing and discovering structure in machine learning. arXiv. https://doi.org/10.48550/arXiv.2111.13693 chicago: Locatello, Francesco. “Enforcing and Discovering Structure in Machine Learning.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2111.13693. ieee: F. Locatello, “Enforcing and discovering structure in machine learning,” arXiv. . ista: Locatello F. Enforcing and discovering structure in machine learning. arXiv, 2111.13693. mla: Locatello, Francesco. “Enforcing and Discovering Structure in Machine Learning.” ArXiv, 2111.13693, doi:10.48550/arXiv.2111.13693. short: F. Locatello, ArXiv (n.d.). date_created: 2023-08-22T14:23:35Z date_published: 2021-11-26T00:00:00Z date_updated: 2024-10-14T12:27:49Z day: '26' department: - _id: FrLo doi: 10.48550/arXiv.2111.13693 extern: '1' external_id: arxiv: - '2111.13693' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2111.13693 month: '11' oa: 1 oa_version: Preprint publication: arXiv publication_status: submitted status: public title: Enforcing and discovering structure in machine learning type: preprint user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14332' abstract: - lang: eng text: Learning data representations that are useful for various downstream tasks is a cornerstone of artificial intelligence. While existing methods are typically evaluated on downstream tasks such as classification or generative image quality, we propose to assess representations through their usefulness in downstream control tasks, such as reaching or pushing objects. By training over 10,000 reinforcement learning policies, we extensively evaluate to what extent different representation properties affect out-of-distribution (OOD) generalization. Finally, we demonstrate zero-shot transfer of these policies from simulation to the real world, without any domain randomization or fine-tuning. This paper aims to establish the first systematic characterization of the usefulness of learned representations for real-world OOD downstream tasks. article_processing_charge: No author: - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Manuel full_name: Wuthrich, Manuel last_name: Wuthrich - first_name: Felix full_name: Widmaier, Felix last_name: Widmaier - first_name: Peter Vincent full_name: Gehler, Peter Vincent last_name: Gehler - first_name: Ole full_name: Winther, Ole last_name: Winther - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer citation: ama: 'Träuble F, Dittadi A, Wuthrich M, et al. Representation learning for out-of-distribution generalization in reinforcement learning. In: ICML 2021 Workshop on Unsupervised Reinforcement Learning. ; 2021.' apa: Träuble, F., Dittadi, A., Wuthrich, M., Widmaier, F., Gehler, P. V., Winther, O., … Bauer, S. (2021). Representation learning for out-of-distribution generalization in reinforcement learning. In ICML 2021 Workshop on Unsupervised Reinforcement Learning. Virtual. chicago: Träuble, Frederik, Andrea Dittadi, Manuel Wuthrich, Felix Widmaier, Peter Vincent Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. “Representation Learning for Out-of-Distribution Generalization in Reinforcement Learning.” In ICML 2021 Workshop on Unsupervised Reinforcement Learning, 2021. ieee: F. Träuble et al., “Representation learning for out-of-distribution generalization in reinforcement learning,” in ICML 2021 Workshop on Unsupervised Reinforcement Learning, Virtual, 2021. ista: 'Träuble F, Dittadi A, Wuthrich M, Widmaier F, Gehler PV, Winther O, Locatello F, Bachem O, Schölkopf B, Bauer S. 2021. Representation learning for out-of-distribution generalization in reinforcement learning. ICML 2021 Workshop on Unsupervised Reinforcement Learning. ICML: International Conference on Machine Learning.' mla: Träuble, Frederik, et al. “Representation Learning for Out-of-Distribution Generalization in Reinforcement Learning.” ICML 2021 Workshop on Unsupervised Reinforcement Learning, 2021. short: F. Träuble, A. Dittadi, M. Wuthrich, F. Widmaier, P.V. Gehler, O. Winther, F. Locatello, O. Bachem, B. Schölkopf, S. Bauer, in:, ICML 2021 Workshop on Unsupervised Reinforcement Learning, 2021. conference: end_date: 2021-07-23 location: Virtual name: 'ICML: International Conference on Machine Learning' start_date: 2021-07-23 date_created: 2023-09-13T12:43:14Z date_published: 2021-07-23T00:00:00Z date_updated: 2023-09-13T12:44:00Z day: '23' department: - _id: FrLo extern: '1' language: - iso: eng month: '07' oa_version: None publication: ICML 2021 Workshop on Unsupervised Reinforcement Learning publication_status: published quality_controlled: '1' status: public title: Representation learning for out-of-distribution generalization in reinforcement learning type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14125' abstract: - lang: eng text: "Motivation: Recent technological advances have led to an increase in the production and availability of single-cell data. The ability to integrate a set of multi-technology measurements would allow the identification of biologically or clinically meaningful observations through the unification of the perspectives afforded by each technology. In most cases, however, profiling technologies consume the used cells and thus pairwise correspondences between datasets are lost. Due to the sheer size single-cell datasets can acquire, scalable algorithms that are able to universally match single-cell measurements carried out in one cell to its corresponding sibling in another technology are needed.\r\nResults: We propose Single-Cell data Integration via Matching (SCIM), a scalable approach to recover such correspondences in two or more technologies. SCIM assumes that cells share a common (low-dimensional) underlying structure and that the underlying cell distribution is approximately constant across technologies. It constructs a technology-invariant latent space using an autoencoder framework with an adversarial objective. Multi-modal datasets are integrated by pairing cells across technologies using a bipartite matching scheme that operates on the low-dimensional latent representations. We evaluate SCIM on a simulated cellular branching process and show that the cell-to-cell matches derived by SCIM reflect the same pseudotime on the simulated dataset. Moreover, we apply our method to two real-world scenarios, a melanoma tumor sample and a human bone marrow sample, where we pair cells from a scRNA dataset to their sibling cells in a CyTOF dataset achieving 90% and 78% cell-matching accuracy for each one of the samples, respectively." article_processing_charge: No article_type: original author: - first_name: Stefan G full_name: Stark, Stefan G last_name: Stark - first_name: Joanna full_name: Ficek, Joanna last_name: Ficek - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Ximena full_name: Bonilla, Ximena last_name: Bonilla - first_name: Stéphane full_name: Chevrier, Stéphane last_name: Chevrier - first_name: Franziska full_name: Singer, Franziska last_name: Singer - first_name: Rudolf full_name: Aebersold, Rudolf last_name: Aebersold - first_name: Faisal S full_name: Al-Quaddoomi, Faisal S last_name: Al-Quaddoomi - first_name: Jonas full_name: Albinus, Jonas last_name: Albinus - first_name: Ilaria full_name: Alborelli, Ilaria last_name: Alborelli - first_name: Sonali full_name: Andani, Sonali last_name: Andani - first_name: Per-Olof full_name: Attinger, Per-Olof last_name: Attinger - first_name: Marina full_name: Bacac, Marina last_name: Bacac - first_name: Daniel full_name: Baumhoer, Daniel last_name: Baumhoer - first_name: Beatrice full_name: Beck-Schimmer, Beatrice last_name: Beck-Schimmer - first_name: Niko full_name: Beerenwinkel, Niko last_name: Beerenwinkel - first_name: Christian full_name: Beisel, Christian last_name: Beisel - first_name: Lara full_name: Bernasconi, Lara last_name: Bernasconi - first_name: Anne full_name: Bertolini, Anne last_name: Bertolini - first_name: Bernd full_name: Bodenmiller, Bernd last_name: Bodenmiller - first_name: Ximena full_name: Bonilla, Ximena last_name: Bonilla - first_name: Ruben full_name: Casanova, Ruben last_name: Casanova - first_name: Stéphane full_name: Chevrier, Stéphane last_name: Chevrier - first_name: Natalia full_name: Chicherova, Natalia last_name: Chicherova - first_name: Maya full_name: D'Costa, Maya last_name: D'Costa - first_name: Esther full_name: Danenberg, Esther last_name: Danenberg - first_name: Natalie full_name: Davidson, Natalie last_name: Davidson - first_name: Monica-Andreea Dră full_name: gan, Monica-Andreea Dră last_name: gan - first_name: Reinhard full_name: Dummer, Reinhard last_name: Dummer - first_name: Stefanie full_name: Engler, Stefanie last_name: Engler - first_name: Martin full_name: Erkens, Martin last_name: Erkens - first_name: Katja full_name: Eschbach, Katja last_name: Eschbach - first_name: Cinzia full_name: Esposito, Cinzia last_name: Esposito - first_name: André full_name: Fedier, André last_name: Fedier - first_name: Pedro full_name: Ferreira, Pedro last_name: Ferreira - first_name: Joanna full_name: Ficek, Joanna last_name: Ficek - first_name: Anja L full_name: Frei, Anja L last_name: Frei - first_name: Bruno full_name: Frey, Bruno last_name: Frey - first_name: Sandra full_name: Goetze, Sandra last_name: Goetze - first_name: Linda full_name: Grob, Linda last_name: Grob - first_name: Gabriele full_name: Gut, Gabriele last_name: Gut - first_name: Detlef full_name: Günther, Detlef last_name: Günther - first_name: Martina full_name: Haberecker, Martina last_name: Haberecker - first_name: Pirmin full_name: Haeuptle, Pirmin last_name: Haeuptle - first_name: Viola full_name: Heinzelmann-Schwarz, Viola last_name: Heinzelmann-Schwarz - first_name: Sylvia full_name: Herter, Sylvia last_name: Herter - first_name: Rene full_name: Holtackers, Rene last_name: Holtackers - first_name: Tamara full_name: Huesser, Tamara last_name: Huesser - first_name: Anja full_name: Irmisch, Anja last_name: Irmisch - first_name: Francis full_name: Jacob, Francis last_name: Jacob - first_name: Andrea full_name: Jacobs, Andrea last_name: Jacobs - first_name: Tim M full_name: Jaeger, Tim M last_name: Jaeger - first_name: Katharina full_name: Jahn, Katharina last_name: Jahn - first_name: Alva R full_name: James, Alva R last_name: James - first_name: Philip M full_name: Jermann, Philip M last_name: Jermann - first_name: André full_name: Kahles, André last_name: Kahles - first_name: Abdullah full_name: Kahraman, Abdullah last_name: Kahraman - first_name: Viktor H full_name: Koelzer, Viktor H last_name: Koelzer - first_name: Werner full_name: Kuebler, Werner last_name: Kuebler - first_name: Jack full_name: Kuipers, Jack last_name: Kuipers - first_name: Christian P full_name: Kunze, Christian P last_name: Kunze - first_name: Christian full_name: Kurzeder, Christian last_name: Kurzeder - first_name: Kjong-Van full_name: Lehmann, Kjong-Van last_name: Lehmann - first_name: Mitchell full_name: Levesque, Mitchell last_name: Levesque - first_name: Sebastian full_name: Lugert, Sebastian last_name: Lugert - first_name: Gerd full_name: Maass, Gerd last_name: Maass - first_name: Markus full_name: Manz, Markus last_name: Manz - first_name: Philipp full_name: Markolin, Philipp last_name: Markolin - first_name: Julien full_name: Mena, Julien last_name: Mena - first_name: Ulrike full_name: Menzel, Ulrike last_name: Menzel - first_name: Julian M full_name: Metzler, Julian M last_name: Metzler - first_name: Nicola full_name: Miglino, Nicola last_name: Miglino - first_name: Emanuela S full_name: Milani, Emanuela S last_name: Milani - first_name: Holger full_name: Moch, Holger last_name: Moch - first_name: Simone full_name: Muenst, Simone last_name: Muenst - first_name: Riccardo full_name: Murri, Riccardo last_name: Murri - first_name: Charlotte KY full_name: Ng, Charlotte KY last_name: Ng - first_name: Stefan full_name: Nicolet, Stefan last_name: Nicolet - first_name: Marta full_name: Nowak, Marta last_name: Nowak - first_name: Patrick GA full_name: Pedrioli, Patrick GA last_name: Pedrioli - first_name: Lucas full_name: Pelkmans, Lucas last_name: Pelkmans - first_name: Salvatore full_name: Piscuoglio, Salvatore last_name: Piscuoglio - first_name: Michael full_name: Prummer, Michael last_name: Prummer - first_name: Mathilde full_name: Ritter, Mathilde last_name: Ritter - first_name: Christian full_name: Rommel, Christian last_name: Rommel - first_name: María L full_name: Rosano-González, María L last_name: Rosano-González - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Natascha full_name: Santacroce, Natascha last_name: Santacroce - first_name: Jacobo Sarabia del full_name: Castillo, Jacobo Sarabia del last_name: Castillo - first_name: Ramona full_name: Schlenker, Ramona last_name: Schlenker - first_name: Petra C full_name: Schwalie, Petra C last_name: Schwalie - first_name: Severin full_name: Schwan, Severin last_name: Schwan - first_name: Tobias full_name: Schär, Tobias last_name: Schär - first_name: Gabriela full_name: Senti, Gabriela last_name: Senti - first_name: Franziska full_name: Singer, Franziska last_name: Singer - first_name: Sujana full_name: Sivapatham, Sujana last_name: Sivapatham - first_name: Berend full_name: Snijder, Berend last_name: Snijder - first_name: Bettina full_name: Sobottka, Bettina last_name: Sobottka - first_name: Vipin T full_name: Sreedharan, Vipin T last_name: Sreedharan - first_name: Stefan full_name: Stark, Stefan last_name: Stark - first_name: Daniel J full_name: Stekhoven, Daniel J last_name: Stekhoven - first_name: Alexandre PA full_name: Theocharides, Alexandre PA last_name: Theocharides - first_name: Tinu M full_name: Thomas, Tinu M last_name: Thomas - first_name: Markus full_name: Tolnay, Markus last_name: Tolnay - first_name: Vinko full_name: Tosevski, Vinko last_name: Tosevski - first_name: Nora C full_name: Toussaint, Nora C last_name: Toussaint - first_name: Mustafa A full_name: Tuncel, Mustafa A last_name: Tuncel - first_name: Marina full_name: Tusup, Marina last_name: Tusup - first_name: Audrey Van full_name: Drogen, Audrey Van last_name: Drogen - first_name: Marcus full_name: Vetter, Marcus last_name: Vetter - first_name: Tatjana full_name: Vlajnic, Tatjana last_name: Vlajnic - first_name: Sandra full_name: Weber, Sandra last_name: Weber - first_name: Walter P full_name: Weber, Walter P last_name: Weber - first_name: Rebekka full_name: Wegmann, Rebekka last_name: Wegmann - first_name: Michael full_name: Weller, Michael last_name: Weller - first_name: Fabian full_name: Wendt, Fabian last_name: Wendt - first_name: Norbert full_name: Wey, Norbert last_name: Wey - first_name: Andreas full_name: Wicki, Andreas last_name: Wicki - first_name: Bernd full_name: Wollscheid, Bernd last_name: Wollscheid - first_name: Shuqing full_name: Yu, Shuqing last_name: Yu - first_name: Johanna full_name: Ziegler, Johanna last_name: Ziegler - first_name: Marc full_name: Zimmermann, Marc last_name: Zimmermann - first_name: Martin full_name: Zoche, Martin last_name: Zoche - first_name: Gregor full_name: Zuend, Gregor last_name: Zuend - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Kjong-Van full_name: Lehmann, Kjong-Van last_name: Lehmann citation: ama: 'Stark SG, Ficek J, Locatello F, et al. SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. 2020;36(Supplement_2):i919-i927. doi:10.1093/bioinformatics/btaa843' apa: 'Stark, S. G., Ficek, J., Locatello, F., Bonilla, X., Chevrier, S., Singer, F., … Lehmann, K.-V. (2020). SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. Oxford University Press. https://doi.org/10.1093/bioinformatics/btaa843' chicago: 'Stark, Stefan G, Joanna Ficek, Francesco Locatello, Ximena Bonilla, Stéphane Chevrier, Franziska Singer, Rudolf Aebersold, et al. “SCIM: Universal Single-Cell Matching with Unpaired Feature Sets.” Bioinformatics. Oxford University Press, 2020. https://doi.org/10.1093/bioinformatics/btaa843.' ieee: 'S. G. Stark et al., “SCIM: Universal single-cell matching with unpaired feature sets,” Bioinformatics, vol. 36, no. Supplement_2. Oxford University Press, pp. i919–i927, 2020.' ista: 'Stark SG et al. 2020. SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. 36(Supplement_2), i919–i927.' mla: 'Stark, Stefan G., et al. “SCIM: Universal Single-Cell Matching with Unpaired Feature Sets.” Bioinformatics, vol. 36, no. Supplement_2, Oxford University Press, 2020, pp. i919–27, doi:10.1093/bioinformatics/btaa843.' short: S.G. Stark, J. Ficek, F. Locatello, X. Bonilla, S. Chevrier, F. Singer, R. Aebersold, F.S. Al-Quaddoomi, J. Albinus, I. Alborelli, S. Andani, P.-O. Attinger, M. Bacac, D. Baumhoer, B. Beck-Schimmer, N. Beerenwinkel, C. Beisel, L. Bernasconi, A. Bertolini, B. Bodenmiller, X. Bonilla, R. Casanova, S. Chevrier, N. Chicherova, M. D’Costa, E. Danenberg, N. Davidson, M.-A.D. gan, R. Dummer, S. Engler, M. Erkens, K. Eschbach, C. Esposito, A. Fedier, P. Ferreira, J. Ficek, A.L. Frei, B. Frey, S. Goetze, L. Grob, G. Gut, D. Günther, M. Haberecker, P. Haeuptle, V. Heinzelmann-Schwarz, S. Herter, R. Holtackers, T. Huesser, A. Irmisch, F. Jacob, A. Jacobs, T.M. Jaeger, K. Jahn, A.R. James, P.M. Jermann, A. Kahles, A. Kahraman, V.H. Koelzer, W. Kuebler, J. Kuipers, C.P. Kunze, C. Kurzeder, K.-V. Lehmann, M. Levesque, S. Lugert, G. Maass, M. Manz, P. Markolin, J. Mena, U. Menzel, J.M. Metzler, N. Miglino, E.S. Milani, H. Moch, S. Muenst, R. Murri, C.K. Ng, S. Nicolet, M. Nowak, P.G. Pedrioli, L. Pelkmans, S. Piscuoglio, M. Prummer, M. Ritter, C. Rommel, M.L. Rosano-González, G. Rätsch, N. Santacroce, J.S. del Castillo, R. Schlenker, P.C. Schwalie, S. Schwan, T. Schär, G. Senti, F. Singer, S. Sivapatham, B. Snijder, B. Sobottka, V.T. Sreedharan, S. Stark, D.J. Stekhoven, A.P. Theocharides, T.M. Thomas, M. Tolnay, V. Tosevski, N.C. Toussaint, M.A. Tuncel, M. Tusup, A.V. Drogen, M. Vetter, T. Vlajnic, S. Weber, W.P. Weber, R. Wegmann, M. Weller, F. Wendt, N. Wey, A. Wicki, B. Wollscheid, S. Yu, J. Ziegler, M. Zimmermann, M. Zoche, G. Zuend, G. Rätsch, K.-V. Lehmann, Bioinformatics 36 (2020) i919–i927. date_created: 2023-08-21T12:28:20Z date_published: 2020-12-01T00:00:00Z date_updated: 2023-09-11T10:21:00Z day: '01' department: - _id: FrLo doi: 10.1093/bioinformatics/btaa843 extern: '1' external_id: pmid: - '33381818' intvolume: ' 36' issue: Supplement_2 keyword: - Computational Mathematics - Computational Theory and Mathematics - Computer Science Applications - Molecular Biology - Biochemistry - Statistics and Probability language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.1093/bioinformatics/btaa843 month: '12' oa: 1 oa_version: Published Version page: i919-i927 pmid: 1 publication: Bioinformatics publication_identifier: eissn: - 1367-4811 publication_status: published publisher: Oxford University Press quality_controlled: '1' related_material: link: - relation: software url: https://github.com/ratschlab/scim scopus_import: '1' status: public title: 'SCIM: Universal single-cell matching with unpaired feature sets' type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 36 year: '2020' ... --- _id: '14186' abstract: - lang: eng text: "The goal of the unsupervised learning of disentangled representations is to\r\nseparate the independent explanatory factors of variation in the data without\r\naccess to supervision. In this paper, we summarize the results of Locatello et\r\nal., 2019, and focus on their implications for practitioners. We discuss the\r\ntheoretical result showing that the unsupervised learning of disentangled\r\nrepresentations is fundamentally impossible without inductive biases and the\r\npractical challenges it entails. Finally, we comment on our experimental\r\nfindings, highlighting the limitations of state-of-the-art approaches and\r\ndirections for future research." article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Mario full_name: Lucic, Mario last_name: Lucic - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Sylvain full_name: Gelly, Sylvain last_name: Gelly - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: 'Locatello F, Bauer S, Lucic M, et al. A commentary on the unsupervised learning of disentangled representations. In: The 34th AAAI Conference on Artificial Intelligence. Vol 34. Association for the Advancement of Artificial Intelligence; 2020:13681-13684. doi:10.1609/aaai.v34i09.7120' apa: 'Locatello, F., Bauer, S., Lucic, M., Rätsch, G., Gelly, S., Schölkopf, B., & Bachem, O. (2020). A commentary on the unsupervised learning of disentangled representations. In The 34th AAAI Conference on Artificial Intelligence (Vol. 34, pp. 13681–13684). New York, NY, United States: Association for the Advancement of Artificial Intelligence. https://doi.org/10.1609/aaai.v34i09.7120' chicago: Locatello, Francesco, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. “A Commentary on the Unsupervised Learning of Disentangled Representations.” In The 34th AAAI Conference on Artificial Intelligence, 34:13681–84. Association for the Advancement of Artificial Intelligence, 2020. https://doi.org/10.1609/aaai.v34i09.7120. ieee: F. Locatello et al., “A commentary on the unsupervised learning of disentangled representations,” in The 34th AAAI Conference on Artificial Intelligence, New York, NY, United States, 2020, vol. 34, no. 9, pp. 13681–13684. ista: 'Locatello F, Bauer S, Lucic M, Rätsch G, Gelly S, Schölkopf B, Bachem O. 2020. A commentary on the unsupervised learning of disentangled representations. The 34th AAAI Conference on Artificial Intelligence. AAAI: Conference on Artificial Intelligence vol. 34, 13681–13684.' mla: Locatello, Francesco, et al. “A Commentary on the Unsupervised Learning of Disentangled Representations.” The 34th AAAI Conference on Artificial Intelligence, vol. 34, no. 9, Association for the Advancement of Artificial Intelligence, 2020, pp. 13681–84, doi:10.1609/aaai.v34i09.7120. short: F. Locatello, S. Bauer, M. Lucic, G. Rätsch, S. Gelly, B. Schölkopf, O. Bachem, in:, The 34th AAAI Conference on Artificial Intelligence, Association for the Advancement of Artificial Intelligence, 2020, pp. 13681–13684. conference: end_date: 2020-02-12 location: New York, NY, United States name: 'AAAI: Conference on Artificial Intelligence' start_date: 2020-02-07 date_created: 2023-08-22T14:07:26Z date_published: 2020-07-28T00:00:00Z date_updated: 2023-09-12T07:44:48Z day: '28' department: - _id: FrLo doi: 10.1609/aaai.v34i09.7120 extern: '1' external_id: arxiv: - '2007.14184' intvolume: ' 34' issue: '9' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2007.14184 month: '07' oa: 1 oa_version: Preprint page: 13681-13684 publication: The 34th AAAI Conference on Artificial Intelligence publication_identifier: eissn: - 2374-3468 isbn: - '9781577358350' publication_status: published publisher: Association for the Advancement of Artificial Intelligence quality_controlled: '1' scopus_import: '1' status: public title: A commentary on the unsupervised learning of disentangled representations type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2020' ...