---
OA_place: repository
OA_type: green
_id: '18956'
abstract:
- lang: eng
text: 'Group Activity Recognition (GAR) aims to detect the activity performed by
multiple actors in a scene. Prior works model the spatio-temporal features based
on the RGB, optical flow or keypoint data types. On the contrary, our hypothesis
is that by only using the RGB data without temporality, the performance can be
maintained with a negligible loss in accuracy. To that end, we propose a novel
GAR technique for volleyball videos, DECOMPL, which consists of two complementary
branches. In the visual branch, it extracts the features using attention pooling.
In the coordinate branch, it considers the configuration of the players and extracts
the spatial information from the box coordinates. Moreover, we analyzed the Volleyball
dataset that the recent literature is mostly based on, and systematically reannotated
it to emphasize the group concept. Experimental results demonstrated the effectiveness
of the proposed model DECOMPL, which delivered the best/second best GAR performance
with the reannotations/original annotations among the comparable state-of-the-art
methods. Code and new annotations are available at GitHub: https://github.com/berkerdemirel/decompl'
article_processing_charge: No
arxiv: 1
author:
- first_name: Berker
full_name: Demirel, Berker
id: 8b4bc47f-3200-11ee-973b-8f0e7be21a9f
last_name: Demirel
- first_name: Huseyin
full_name: Ozkan, Huseyin
last_name: Ozkan
citation:
ama: 'Demirel B, Ozkan H. Decompl: Decompositional learning with attention pooling
for group activity recognition from a single volleyball image. In: 2024 IEEE
International Conference on Image Processing. IEEE; 2024:977-983. doi:10.1109/icip51287.2024.10647499'
apa: 'Demirel, B., & Ozkan, H. (2024). Decompl: Decompositional learning with
attention pooling for group activity recognition from a single volleyball image.
In 2024 IEEE International Conference on Image Processing (pp. 977–983).
Abu Dhabi, United Arab Emirates: IEEE. https://doi.org/10.1109/icip51287.2024.10647499'
chicago: 'Demirel, Berker, and Huseyin Ozkan. “Decompl: Decompositional Learning
with Attention Pooling for Group Activity Recognition from a Single Volleyball
Image.” In 2024 IEEE International Conference on Image Processing, 977–83.
IEEE, 2024. https://doi.org/10.1109/icip51287.2024.10647499.'
ieee: 'B. Demirel and H. Ozkan, “Decompl: Decompositional learning with attention
pooling for group activity recognition from a single volleyball image,” in 2024
IEEE International Conference on Image Processing, Abu Dhabi, United Arab
Emirates, 2024, pp. 977–983.'
ista: 'Demirel B, Ozkan H. 2024. Decompl: Decompositional learning with attention
pooling for group activity recognition from a single volleyball image. 2024 IEEE
International Conference on Image Processing. ICIP: International Conference on
Image Processing, 977–983.'
mla: 'Demirel, Berker, and Huseyin Ozkan. “Decompl: Decompositional Learning with
Attention Pooling for Group Activity Recognition from a Single Volleyball Image.”
2024 IEEE International Conference on Image Processing, IEEE, 2024, pp.
977–83, doi:10.1109/icip51287.2024.10647499.'
short: B. Demirel, H. Ozkan, in:, 2024 IEEE International Conference on Image Processing,
IEEE, 2024, pp. 977–983.
conference:
end_date: 2024-10-30
location: Abu Dhabi, United Arab Emirates
name: 'ICIP: International Conference on Image Processing'
start_date: 2024-10-27
corr_author: '1'
date_created: 2025-01-29T12:22:24Z
date_published: 2024-11-01T00:00:00Z
date_updated: 2025-09-09T12:13:12Z
day: '01'
department:
- _id: FrLo
doi: 10.1109/icip51287.2024.10647499
external_id:
arxiv:
- '2303.06439'
isi:
- '001442947000143'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2303.06439
month: '11'
oa: 1
oa_version: Preprint
page: 977-983
publication: 2024 IEEE International Conference on Image Processing
publication_identifier:
eisbn:
- '9798350349399'
eissn:
- 2381-8549
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/berkerdemirel/decompl
status: public
title: 'Decompl: Decompositional learning with attention pooling for group activity
recognition from a single volleyball image'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18964'
abstract:
- lang: eng
text: Object-centric learning (OCL) extracts the representation of objects with
slots, offering an exceptional blend of flexibility and interpretability for abstracting
low-level perceptual features. A widely adopted method within OCL is slot attention,
which utilizes attention mechanisms to iteratively refine slot representations.
However, a major draw-back of most object-centric models, including slot attention,
is their reliance on predefining the number of slots. This not only necessitates
prior knowledge of the dataset but also overlooks the inherent variability in
the number of objects present in each instance. To overcome this fundamental limitation,
we present a novel complexity-aware object auto-encoder framework. Within this
framework, we introduce an adaptive slot attention (AdaSlot) mecha-nism that dynamically
determines the optimal number of slots based on the content of the data. This
is achieved by proposing a discrete slot sampling module that is responsible for
selecting an appropriate number of slots from a candidate list. Furthermore, we
introduce a masked slot decoder that suppresses unselected slots during the decoding
process. Our framework, tested extensively on object discovery tasks with various
datasets, shows performance matching or exceeding top fixed-slot models. Moreover,
our analysis substantiates that our method exhibits the capability to dynamically
adapt the slot number according to each instance's complexity, offering the potential
for further exploration in slot attention research. Project will be available
at https://kfan21.github.io/AdaSlot/
acknowledgement: Yanwei Fu is the corresponding authour. Yanwei Fu is with School
of Data Science, Fudan University, Shanghai Key Lab of Intelligent Information Processing,
Fudan University, and Fudan ISTBI-ZJNU Algorithm Centre for Brain-inspired Intelligence,
Zhejiang Normal University, Jinhua, China.
article_processing_charge: No
arxiv: 1
author:
- first_name: Ke
full_name: Fan, Ke
last_name: Fan
- first_name: Zechen
full_name: Bai, Zechen
last_name: Bai
- first_name: Tianjun
full_name: Xiao, Tianjun
last_name: Xiao
- first_name: Tong
full_name: He, Tong
last_name: He
- first_name: Max
full_name: Horn, Max
last_name: Horn
- first_name: Yanwei
full_name: Fu, Yanwei
last_name: Fu
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Zheng
full_name: Zhang, Zheng
last_name: Zhang
citation:
ama: 'Fan K, Bai Z, Xiao T, et al. Adaptive slot attention: Object discovery with
dynamic slot number. In: 2024 IEEE/CVF Conference on Computer Vision and Pattern
Recognition. IEEE; 2024. doi:10.1109/cvpr52733.2024.02176'
apa: 'Fan, K., Bai, Z., Xiao, T., He, T., Horn, M., Fu, Y., … Zhang, Z. (2024).
Adaptive slot attention: Object discovery with dynamic slot number. In 2024
IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA,
United States: IEEE. https://doi.org/10.1109/cvpr52733.2024.02176'
chicago: 'Fan, Ke, Zechen Bai, Tianjun Xiao, Tong He, Max Horn, Yanwei Fu, Francesco
Locatello, and Zheng Zhang. “Adaptive Slot Attention: Object Discovery with Dynamic
Slot Number.” In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
IEEE, 2024. https://doi.org/10.1109/cvpr52733.2024.02176.'
ieee: 'K. Fan et al., “Adaptive slot attention: Object discovery with dynamic
slot number,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition,
Seattle, WA, United States, 2024.'
ista: 'Fan K, Bai Z, Xiao T, He T, Horn M, Fu Y, Locatello F, Zhang Z. 2024. Adaptive
slot attention: Object discovery with dynamic slot number. 2024 IEEE/CVF Conference
on Computer Vision and Pattern Recognition. CVPR: Conference on Computer Vision
and Pattern Recognition.'
mla: 'Fan, Ke, et al. “Adaptive Slot Attention: Object Discovery with Dynamic Slot
Number.” 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition,
IEEE, 2024, doi:10.1109/cvpr52733.2024.02176.'
short: K. Fan, Z. Bai, T. Xiao, T. He, M. Horn, Y. Fu, F. Locatello, Z. Zhang, in:,
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2024.
conference:
end_date: 2024-06-22
location: Seattle, WA, United States
name: 'CVPR: Conference on Computer Vision and Pattern Recognition'
start_date: 2024-06-16
date_created: 2025-01-29T14:27:39Z
date_published: 2024-06-15T00:00:00Z
date_updated: 2025-09-09T12:15:17Z
day: '15'
department:
- _id: FrLo
doi: 10.1109/cvpr52733.2024.02176
external_id:
arxiv:
- '2406.09196'
isi:
- '001342515506043'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2406.09196
month: '06'
oa: 1
oa_version: Preprint
publication: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition
publication_identifier:
eisbn:
- '9798350353006'
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://kfan21.github.io/AdaSlot/
status: public
title: 'Adaptive slot attention: Object discovery with dynamic slot number'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18971'
abstract:
- lang: eng
text: 'Models prone to spurious correlations in training data often produce brittle
predictions and introduce unintended biases. Addressing this challenge typically
involves methods relying on prior knowledge and group annotation to remove spurious
correlations, which may not be readily available in many applications. In this
paper, we establish a novel connection between unsupervised object-centric learning
and mitigation of spurious correlations. Instead of directly inferring subgroups
with varying correlations with labels, our approach focuses on discovering concepts:
discrete ideas that are shared across input samples. Leveraging existing object-centric
representation learning, we introduce CoBalT: a concept balancing technique that
effectively mitigates spurious correlations without requiring human labeling of
subgroups. Evaluation across the benchmark datasets for sub-population shifts
demonstrate superior or competitive performance compared state-of-the-art baselines,
without the need for group annotation. Code is available at https://github.com/rarefin/CoBalT'
acknowledgement: "We acknowledge the support of the Canada CIFAR AI Chair Program
and IVADO. We thank Mila and Compute Canada for providing computational resources.\r\n"
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Rifat
full_name: Arefin, Rifat
last_name: Arefin
- first_name: Yan
full_name: Zhang, Yan
last_name: Zhang
- first_name: Aristide
full_name: Baratin, Aristide
last_name: Baratin
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Irina
full_name: Rish, Irina
last_name: Rish
- first_name: Dianbo
full_name: Liu, Dianbo
last_name: Liu
- first_name: Kenji
full_name: Kawaguchi, Kenji
last_name: Kawaguchi
citation:
ama: 'Arefin R, Zhang Y, Baratin A, et al. Unsupervised concept discovery mitigates
spurious correlations. In: Proceedings of the 41st International Conference
on Machine Learning. Vol 235. ML Research Press; 2024:1672-1688.'
apa: 'Arefin, R., Zhang, Y., Baratin, A., Locatello, F., Rish, I., Liu, D., &
Kawaguchi, K. (2024). Unsupervised concept discovery mitigates spurious correlations.
In Proceedings of the 41st International Conference on Machine Learning
(Vol. 235, pp. 1672–1688). Vienna, Austria: ML Research Press.'
chicago: Arefin, Rifat, Yan Zhang, Aristide Baratin, Francesco Locatello, Irina
Rish, Dianbo Liu, and Kenji Kawaguchi. “Unsupervised Concept Discovery Mitigates
Spurious Correlations.” In Proceedings of the 41st International Conference
on Machine Learning, 235:1672–88. ML Research Press, 2024.
ieee: R. Arefin et al., “Unsupervised concept discovery mitigates spurious
correlations,” in Proceedings of the 41st International Conference on Machine
Learning, Vienna, Austria, 2024, vol. 235, pp. 1672–1688.
ista: 'Arefin R, Zhang Y, Baratin A, Locatello F, Rish I, Liu D, Kawaguchi K. 2024.
Unsupervised concept discovery mitigates spurious correlations. Proceedings of
the 41st International Conference on Machine Learning. ICML: International Conference
on Machine Learning, PMLR, vol. 235, 1672–1688.'
mla: Arefin, Rifat, et al. “Unsupervised Concept Discovery Mitigates Spurious Correlations.”
Proceedings of the 41st International Conference on Machine Learning, vol.
235, ML Research Press, 2024, pp. 1672–88.
short: R. Arefin, Y. Zhang, A. Baratin, F. Locatello, I. Rish, D. Liu, K. Kawaguchi,
in:, Proceedings of the 41st International Conference on Machine Learning, ML
Research Press, 2024, pp. 1672–1688.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
date_created: 2025-01-30T07:21:57Z
date_published: 2024-07-30T00:00:00Z
date_updated: 2025-01-30T07:23:10Z
day: '30'
department:
- _id: FrLo
external_id:
arxiv:
- '2402.13368'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2402.13368
month: '07'
oa: 1
oa_version: Preprint
page: 1672-1688
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/rarefin/CoBalT
scopus_import: '1'
status: public
title: Unsupervised concept discovery mitigates spurious correlations
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 235
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '18996'
abstract:
- lang: eng
text: 'We consider the linear causal representation learning setting where we observe
a linear mixing of d unknown latent factors, which follow a linear structural
causal model. Recent work has shown that it is possible to recover the latent
factors as well as the underlying structural causal model over them, up to permutation
and scaling, provided that we have at least d environments, each of which corresponds
to perfect interventions on a single latent node (factor). After this powerful
result, a key open problem faced by the community has been to relax these conditions:
allow for coarser than perfect single-node interventions, and allow for fewer
than d of them, since the number of latent factors d could be very large. In this
work, we consider precisely such a setting, where we allow a smaller than d number
of environments, and also allow for very coarse interventions that can very coarsely
\textit{change the entire causal graph over the latent factors}. On the flip side,
we relax what we wish to extract to simply the \textit{list of nodes that have
shifted between one or more environments}. We provide a surprising identifiability
result that it is indeed possible, under some very mild standard assumptions,
to identify the set of shifted nodes. Our identifiability proof moreover is a
constructive one: we explicitly provide necessary and sufficient conditions for
a node to be a shifted node, and show that we can check these conditions given
observed data. Our algorithm lends itself very naturally to the sample setting
where instead of just interventional distributions, we are provided datasets of
samples from each of these distributions. We corroborate our results on both synthetic
experiments as well as an interesting psychometric dataset. The code can be found
at https://github.com/TianyuCodings/iLCS.'
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Tianyu
full_name: Chen, Tianyu
last_name: Chen
- first_name: Kevin
full_name: Bello, Kevin
last_name: Bello
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Bryon
full_name: Aragam, Bryon
last_name: Aragam
- first_name: Pradeep Kumar
full_name: Ravikumar, Pradeep Kumar
last_name: Ravikumar
citation:
ama: 'Chen T, Bello K, Locatello F, Aragam B, Ravikumar PK. Identifying general
mechanism shifts in linear causal representations. In: 38th Conference on Neural
Information Processing Systems. Vol 37. Neural Information Processing Systems
Foundation; 2024.'
apa: 'Chen, T., Bello, K., Locatello, F., Aragam, B., & Ravikumar, P. K. (2024).
Identifying general mechanism shifts in linear causal representations. In 38th
Conference on Neural Information Processing Systems (Vol. 37). Vancouver,
Canada: Neural Information Processing Systems Foundation.'
chicago: Chen, Tianyu, Kevin Bello, Francesco Locatello, Bryon Aragam, and Pradeep
Kumar Ravikumar. “Identifying General Mechanism Shifts in Linear Causal Representations.”
In 38th Conference on Neural Information Processing Systems, Vol. 37. Neural
Information Processing Systems Foundation, 2024.
ieee: T. Chen, K. Bello, F. Locatello, B. Aragam, and P. K. Ravikumar, “Identifying
general mechanism shifts in linear causal representations,” in 38th Conference
on Neural Information Processing Systems, Vancouver, Canada, 2024, vol. 37.
ista: 'Chen T, Bello K, Locatello F, Aragam B, Ravikumar PK. 2024. Identifying general
mechanism shifts in linear causal representations. 38th Conference on Neural Information
Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in
Neural Information Processing Systems, vol. 37.'
mla: Chen, Tianyu, et al. “Identifying General Mechanism Shifts in Linear Causal
Representations.” 38th Conference on Neural Information Processing Systems,
vol. 37, Neural Information Processing Systems Foundation, 2024.
short: T. Chen, K. Bello, F. Locatello, B. Aragam, P.K. Ravikumar, in:, 38th Conference
on Neural Information Processing Systems, Neural Information Processing Systems
Foundation, 2024.
conference:
end_date: 2024-12-16
location: Vancouver, Canada
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2024-12-16
date_created: 2025-02-04T13:09:34Z
date_published: 2024-09-25T00:00:00Z
date_updated: 2025-07-07T13:23:49Z
day: '25'
ddc:
- '000'
department:
- _id: FrLo
external_id:
arxiv:
- '2410.24059'
file:
- access_level: open_access
checksum: 75c3091e70bd2916cd94afbf40a0c425
content_type: application/pdf
creator: dernst
date_created: 2025-02-04T13:09:08Z
date_updated: 2025-02-04T13:09:08Z
file_id: '18997'
file_name: 2024_NeurIPS_Chen.pdf
file_size: 5659119
relation: main_file
success: 1
file_date_updated: 2025-02-04T13:09:08Z
has_accepted_license: '1'
intvolume: ' 37'
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
month: '09'
oa: 1
oa_version: Published Version
publication: 38th Conference on Neural Information Processing Systems
publication_identifier:
eissn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: Identifying general mechanism shifts in linear causal representations
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 37
year: '2024'
...
---
OA_place: publisher
OA_type: gold
_id: '19005'
abstract:
- lang: eng
text: "Causal representation learning promises to extend causal models to hidden
causal\r\nvariables from raw entangled measurements. However, most progress has
focused\r\non proving identifiability results in different settings, and we are
not aware of any\r\nsuccessful real-world application. At the same time, the field
of dynamical systems\r\nbenefited from deep learning and scaled to countless applications
but does not allow\r\nparameter identification. In this paper, we draw a clear
connection between the two\r\nand their key assumptions, allowing us to apply
identifiable methods developed\r\nin causal representation learning to dynamical
systems. At the same time, we can\r\nleverage scalable differentiable solvers
developed for differential equations to build\r\nmodels that are both identifiable
and practical. Overall, we learn explicitly controllable models that isolate the
trajectory-specific parameters for further downstream\r\ntasks such as out-of-distribution
classification or treatment effect estimation. We\r\nexperiment with a wind simulator
with partially known factors of variation. We\r\nalso apply the resulting model
to real-world climate data and successfully answer\r\ndownstream causal questions
in line with existing literature on climate change.\r\nCode is available at https://github.com/CausalLearningAI/crl-dynamical-systems."
acknowledgement: "We thank Niklas Boers for recommending the SpeedyWeather simulator
and Valentino Maiorca\r\nfor guidance on Fourier transformation for SST data. We
are also grateful to Shimeng Huang and Riccardo Cadei for their feedback on the
treatment effect estimation experiment and to Jiale Chen and Adeel Pervez for their
assistance with the solver implementation. Finally, we appreciate the anonymous
reviewers for their insightful suggestions, which helped improve the manuscript. "
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Dingling
full_name: Yao, Dingling
id: d3e02e50-48a8-11ee-8f62-c108061797fa
last_name: Yao
- first_name: Caroline J
full_name: Muller, Caroline J
id: f978ccb0-3f7f-11eb-b193-b0e2bd13182b
last_name: Muller
orcid: 0000-0001-5836-5350
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Yao D, Muller CJ, Locatello F. Marrying causal representation learning with
dynamical systems for science. In: 38th Conference on Neural Information Processing
Systems. Vol 37. Neural Information Processing Systems Foundation; 2024.'
apa: 'Yao, D., Muller, C. J., & Locatello, F. (2024). Marrying causal representation
learning with dynamical systems for science. In 38th Conference on Neural Information
Processing Systems (Vol. 37). Vancouver, Canada: Neural Information Processing
Systems Foundation.'
chicago: Yao, Dingling, Caroline J Muller, and Francesco Locatello. “Marrying Causal
Representation Learning with Dynamical Systems for Science.” In 38th Conference
on Neural Information Processing Systems, Vol. 37. Neural Information Processing
Systems Foundation, 2024.
ieee: D. Yao, C. J. Muller, and F. Locatello, “Marrying causal representation learning
with dynamical systems for science,” in 38th Conference on Neural Information
Processing Systems, Vancouver, Canada, 2024, vol. 37.
ista: 'Yao D, Muller CJ, Locatello F. 2024. Marrying causal representation learning
with dynamical systems for science. 38th Conference on Neural Information Processing
Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 37.'
mla: Yao, Dingling, et al. “Marrying Causal Representation Learning with Dynamical
Systems for Science.” 38th Conference on Neural Information Processing Systems,
vol. 37, Neural Information Processing Systems Foundation, 2024.
short: D. Yao, C.J. Muller, F. Locatello, in:, 38th Conference on Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2024.
conference:
end_date: 2024-12-16
location: Vancouver, Canada
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2024-12-16
corr_author: '1'
date_created: 2025-02-05T07:49:00Z
date_published: 2024-12-01T00:00:00Z
date_updated: 2025-07-10T11:51:32Z
day: '01'
ddc:
- '000'
- '550'
department:
- _id: CaMu
- _id: FrLo
external_id:
arxiv:
- '2405.13888'
file:
- access_level: open_access
checksum: fe8832367e7143876f178244385d859e
content_type: application/pdf
creator: dernst
date_created: 2025-02-05T07:44:58Z
date_updated: 2025-02-05T07:44:58Z
file_id: '19006'
file_name: 2024_NeurIPS_Yao.pdf
file_size: 2595855
relation: main_file
success: 1
file_date_updated: 2025-02-05T07:44:58Z
has_accepted_license: '1'
intvolume: ' 37'
language:
- iso: eng
month: '12'
oa: 1
oa_version: Published Version
publication: 38th Conference on Neural Information Processing Systems
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/CausalLearningAI/crl-dynamical-systems
scopus_import: '1'
status: public
title: Marrying causal representation learning with dynamical systems for science
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 37
year: '2024'
...
---
OA_place: publisher
OA_type: hybrid
_id: '19007'
abstract:
- lang: eng
text: "Learning modular object-centric representations is crucial for systematic
generalization. Existing methods show promising object-binding capabilities empirically,\r\nbut
theoretical identifiability guarantees remain relatively underdeveloped. Understanding
when object-centric representations can theoretically be identified is\r\ncrucial
for scaling slot-based methods to high-dimensional images with correctness\r\nguarantees.
To that end, we propose a probabilistic slot-attention algorithm that\r\nimposes
an aggregate mixture prior over object-centric slot representations, thereby\r\nproviding
slot identifiability guarantees without supervision, up to an equivalence\r\nrelation.
We provide empirical verification of our theoretical identifiability result\r\nusing
both simple 2-dimensional data and high-resolution imaging datasets.\r\n"
acknowledgement: A. Kori is supported by UKRI (grant number EP/S023356/1), as part
of the UKRI Centre for Doctoral Training in Safe and Trusted AI. B. Glocker and
F.D.S. Ribeiro acknowledge the support of the UKRI AI programme, and the Engineering
and Physical Sciences Research Council, for CHAI - EPSRC Causality in Healthcare
AI Hub (grant number EP/Y028856/1).
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Avinash
full_name: Kori, Avinash
last_name: Kori
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Ainkaran
full_name: Santhirasekaram, Ainkaran
last_name: Santhirasekaram
- first_name: Francesca
full_name: Toni, Francesca
last_name: Toni
- first_name: Ben
full_name: Glocker, Ben
last_name: Glocker
- first_name: Fabio
full_name: De Sousa Ribeiro, Fabio
last_name: De Sousa Ribeiro
citation:
ama: 'Kori A, Locatello F, Santhirasekaram A, Toni F, Glocker B, De Sousa Ribeiro
F. Identifiable object-centric representation learning via probabilistic slot
attention. In: 38th Conference on Neural Information Processing Systems.
Vol 37. Neural Information Processing Systems Foundation; 2024.'
apa: 'Kori, A., Locatello, F., Santhirasekaram, A., Toni, F., Glocker, B., &
De Sousa Ribeiro, F. (2024). Identifiable object-centric representation learning
via probabilistic slot attention. In 38th Conference on Neural Information
Processing Systems (Vol. 37). Vancouver, Canada: Neural Information Processing
Systems Foundation.'
chicago: Kori, Avinash, Francesco Locatello, Ainkaran Santhirasekaram, Francesca
Toni, Ben Glocker, and Fabio De Sousa Ribeiro. “Identifiable Object-Centric Representation
Learning via Probabilistic Slot Attention.” In 38th Conference on Neural Information
Processing Systems, Vol. 37. Neural Information Processing Systems Foundation,
2024.
ieee: A. Kori, F. Locatello, A. Santhirasekaram, F. Toni, B. Glocker, and F. De
Sousa Ribeiro, “Identifiable object-centric representation learning via probabilistic
slot attention,” in 38th Conference on Neural Information Processing Systems,
Vancouver, Canada, 2024, vol. 37.
ista: 'Kori A, Locatello F, Santhirasekaram A, Toni F, Glocker B, De Sousa Ribeiro
F. 2024. Identifiable object-centric representation learning via probabilistic
slot attention. 38th Conference on Neural Information Processing Systems. NeurIPS:
Neural Information Processing Systems, Advances in Neural Information Processing
Systems, vol. 37.'
mla: Kori, Avinash, et al. “Identifiable Object-Centric Representation Learning
via Probabilistic Slot Attention.” 38th Conference on Neural Information Processing
Systems, vol. 37, Neural Information Processing Systems Foundation, 2024.
short: A. Kori, F. Locatello, A. Santhirasekaram, F. Toni, B. Glocker, F. De Sousa
Ribeiro, in:, 38th Conference on Neural Information Processing Systems, Neural
Information Processing Systems Foundation, 2024.
conference:
end_date: 2024-12-16
location: Vancouver, Canada
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2024-12-16
date_created: 2025-02-05T08:36:22Z
date_published: 2024-12-01T00:00:00Z
date_updated: 2025-05-14T11:29:10Z
day: '01'
ddc:
- '000'
department:
- _id: FrLo
external_id:
arxiv:
- '2406.07141'
file:
- access_level: open_access
checksum: d27b3c7102adc28e798fe41001f0b919
content_type: application/pdf
creator: dernst
date_created: 2025-02-05T08:34:25Z
date_updated: 2025-02-05T08:34:25Z
file_id: '19008'
file_name: 2024_NeurIPS_Kori.pdf
file_size: 6943800
relation: main_file
success: 1
file_date_updated: 2025-02-05T08:34:25Z
has_accepted_license: '1'
intvolume: ' 37'
language:
- iso: eng
month: '12'
oa: 1
oa_version: Published Version
publication: 38th Conference on Neural Information Processing Systems
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: Identifiable object-centric representation learning via probabilistic slot
attention
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 37
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '19515'
abstract:
- lang: eng
text: "Neural models learn data representations that lie on low-dimensional manifolds,\r\nyet
modeling the relation between these representational spaces is an ongoing challenge.
By integrating spectral geometry principles into neural modeling, we show\r\nthat
this problem can be better addressed in the functional domain, mitigating complexity,
while enhancing interpretability and performances on downstream tasks.\r\nTo this
end, we introduce a multi-purpose framework to the representation learning\r\ncommunity,
which allows to: (i) compare different spaces in an interpretable way\r\nand measure
their intrinsic similarity; (ii) find correspondences between them, both\r\nin
unsupervised and weakly supervised settings, and (iii) to effectively transfer\r\nrepresentations
between distinct spaces. We validate our framework on various\r\napplications,
ranging from stitching to retrieval tasks, and on multiple modalities,\r\ndemonstrating
that Latent Functional Maps can serve as a swiss-army knife for\r\nrepresentation
alignment"
acknowledgement: MF is supported by the MSCA IST-Bridge fellowship which has received
funding from the European Union’s Horizon 2020 research and innovation program under
the Marie Skłodowska-Curie grant agreement No 101034413. ER and VM are supported
by the PNRR MUR project PE0000013-FAIR. MP is supported by the Sapienza grant "Predicting
and Explaining Clinical Trial Outcomes", prot. RG12218166FA3F13.
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Marco
full_name: Fumero, Marco
id: 1c1593eb-393f-11ef-bb8e-ab4f1e979650
last_name: Fumero
- first_name: Marco
full_name: Pegoraro, Marco
last_name: Pegoraro
- first_name: Valentino
full_name: Maiorca, Valentino
last_name: Maiorca
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Emanuele
full_name: Rodolà, Emanuele
last_name: Rodolà
citation:
ama: 'Fumero M, Pegoraro M, Maiorca V, Locatello F, Rodolà E. Latent functional
maps: A spectral framework for representation alignment. In: 38th Conference
on Neural Information Processing Systems. Vol 37. Neural Information Processing
Systems Foundation; 2024.'
apa: 'Fumero, M., Pegoraro, M., Maiorca, V., Locatello, F., & Rodolà, E. (2024).
Latent functional maps: A spectral framework for representation alignment. In
38th Conference on Neural Information Processing Systems (Vol. 37). Vancouver,
Canada: Neural Information Processing Systems Foundation.'
chicago: 'Fumero, Marco, Marco Pegoraro, Valentino Maiorca, Francesco Locatello,
and Emanuele Rodolà. “Latent Functional Maps: A Spectral Framework for Representation
Alignment.” In 38th Conference on Neural Information Processing Systems,
Vol. 37. Neural Information Processing Systems Foundation, 2024.'
ieee: 'M. Fumero, M. Pegoraro, V. Maiorca, F. Locatello, and E. Rodolà, “Latent
functional maps: A spectral framework for representation alignment,” in 38th
Conference on Neural Information Processing Systems, Vancouver, Canada, 2024,
vol. 37.'
ista: 'Fumero M, Pegoraro M, Maiorca V, Locatello F, Rodolà E. 2024. Latent functional
maps: A spectral framework for representation alignment. 38th Conference on Neural
Information Processing Systems. NeurIPS: Neural Information Processing Systems,
Advances in Neural Information Processing Systems, vol. 37.'
mla: 'Fumero, Marco, et al. “Latent Functional Maps: A Spectral Framework for Representation
Alignment.” 38th Conference on Neural Information Processing Systems, vol.
37, Neural Information Processing Systems Foundation, 2024.'
short: M. Fumero, M. Pegoraro, V. Maiorca, F. Locatello, E. Rodolà, in:, 38th Conference
on Neural Information Processing Systems, Neural Information Processing Systems
Foundation, 2024.
conference:
end_date: 2024-12-15
location: Vancouver, Canada
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2024-12-09
corr_author: '1'
date_created: 2025-04-06T22:01:32Z
date_published: 2024-12-20T00:00:00Z
date_updated: 2025-05-14T11:36:51Z
day: '20'
department:
- _id: FrLo
ec_funded: 1
external_id:
arxiv:
- '2406.14183'
intvolume: ' 37'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2406.14183
month: '12'
oa: 1
oa_version: Preprint
project:
- _id: fc2ed2f7-9c52-11eb-aca3-c01059dda49c
call_identifier: H2020
grant_number: '101034413'
name: 'IST-BRIDGE: International postdoctoral program'
publication: 38th Conference on Neural Information Processing Systems
publication_identifier:
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'Latent functional maps: A spectral framework for representation alignment'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 37
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '19517'
abstract:
- lang: eng
text: "In this paper, we present a novel data-free method for merging neural networks
in weight space. Differently from most existing works, our method optimizes for
the permutations of network neurons globally across all layers. This allows us
to enforce cycle consistency of the permutations when merging n ≥ 3 models, allowing
circular compositions of permutations to be computed without accumulating error
along the path. We qualitatively and quantitatively motivate the need for such
a constraint, showing its benefits when merging sets of models in scenarios spanning
varying architectures and datasets. We finally show that, when coupled\r\nwith
activation renormalization, our approach yields the best results in the task."
acknowledgement: "This work is supported by the ERC grant no.802554 (SPECGEO), PRIN
2020 project\r\nno.2020TA3K9N (LEGO.AI), and PNRR MUR project PE0000013-FAIR. Marco
Fumero is supported by the MSCA IST-Bridge fellowship which has received funding
from the European Union’s Horizon 2020 research and innovation program under the
Marie Skłodowska-Curie grant agreement No 101034413. We thank Simone Scardapane
for the helpful feedback on the paper."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Donato
full_name: Crisostomi, Donato
last_name: Crisostomi
- first_name: Marco
full_name: Fumero, Marco
id: 1c1593eb-393f-11ef-bb8e-ab4f1e979650
last_name: Fumero
- first_name: Daniele
full_name: Baieri, Daniele
last_name: Baieri
- first_name: Florian
full_name: Bernard, Florian
last_name: Bernard
- first_name: Emanuele
full_name: Rodolà, Emanuele
last_name: Rodolà
citation:
ama: 'Crisostomi D, Fumero M, Baieri D, Bernard F, Rodolà E. C2M3: Cycle-consistent
multi-model merging. In: 38th Conference on Neural Information Processing Systems.
Vol 37. Neural Information Processing Systems Foundation; 2024.'
apa: 'Crisostomi, D., Fumero, M., Baieri, D., Bernard, F., & Rodolà, E. (2024).
C2M3: Cycle-consistent multi-model merging. In 38th Conference on Neural Information
Processing Systems (Vol. 37). Vancouver, Canada: Neural Information Processing
Systems Foundation.'
chicago: 'Crisostomi, Donato, Marco Fumero, Daniele Baieri, Florian Bernard, and
Emanuele Rodolà. “C2M3: Cycle-Consistent Multi-Model Merging.” In 38th Conference
on Neural Information Processing Systems, Vol. 37. Neural Information Processing
Systems Foundation, 2024.'
ieee: 'D. Crisostomi, M. Fumero, D. Baieri, F. Bernard, and E. Rodolà, “C2M3: Cycle-consistent
multi-model merging,” in 38th Conference on Neural Information Processing Systems,
Vancouver, Canada, 2024, vol. 37.'
ista: 'Crisostomi D, Fumero M, Baieri D, Bernard F, Rodolà E. 2024. C2M3: Cycle-consistent
multi-model merging. 38th Conference on Neural Information Processing Systems.
NeurIPS: Neural Information Processing Systems, Advances in Neural Information
Processing Systems, vol. 37.'
mla: 'Crisostomi, Donato, et al. “C2M3: Cycle-Consistent Multi-Model Merging.” 38th
Conference on Neural Information Processing Systems, vol. 37, Neural Information
Processing Systems Foundation, 2024.'
short: D. Crisostomi, M. Fumero, D. Baieri, F. Bernard, E. Rodolà, in:, 38th Conference
on Neural Information Processing Systems, Neural Information Processing Systems
Foundation, 2024.
conference:
end_date: 2024-12-15
location: Vancouver, Canada
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2024-12-09
corr_author: '1'
date_created: 2025-04-06T22:01:32Z
date_published: 2024-12-20T00:00:00Z
date_updated: 2025-05-14T11:36:59Z
day: '20'
department:
- _id: FrLo
ec_funded: 1
external_id:
arxiv:
- '2405.17897'
intvolume: ' 37'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2405.17897
month: '12'
oa: 1
oa_version: Preprint
project:
- _id: fc2ed2f7-9c52-11eb-aca3-c01059dda49c
call_identifier: H2020
grant_number: '101034413'
name: 'IST-BRIDGE: International postdoctoral program'
publication: 38th Conference on Neural Information Processing Systems
publication_identifier:
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'C2M3: Cycle-consistent multi-model merging'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 37
year: '2024'
...
---
OA_place: publisher
OA_type: diamond
_id: '18114'
abstract:
- lang: eng
text: This paper presents Mechanistic Neural Networks, a neural network design for
machine learning applications in the sciences. It incorporates a new Mechanistic
Block in standard architectures to explicitly learn governing differential equations
as representations, revealing the underlying dynamics of data and enhancing interpretability
and efficiency in data modeling. Central to our approach is a novel Relaxed Linear
Programming Solver (NeuRLP) inspired by a technique that reduces solving linear
ODEs to solving linear programs. This integrates well with neural networks and
surpasses the limitations of traditional ODE solvers enabling scalable GPU parallel
processing. Overall, Mechanistic Neural Networks demonstrate their versatility
for scientific machine learning applications, adeptly managing tasks from equation
discovery to dynamic systems modeling. We prove their comprehensive capabilities
in analyzing and interpreting complex scientific data across various applications,
showing significant performance against specialized state-of-the-art methods.
Source code is available at https://github.com/alpz/mech-nn.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Adeel A
full_name: Pervez, Adeel A
id: fca6d90c-d47f-11ee-bc87-93ff51604981
last_name: Pervez
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Efstratios
full_name: Gavves, Efstratios
last_name: Gavves
citation:
ama: 'Pervez AA, Locatello F, Gavves E. Mechanistic neural networks for scientific
machine learning. In: Proceedings of the 41st International Conference on Machine
Learning. Vol 235. ML Research Press; 2024:40484-40501.'
apa: 'Pervez, A. A., Locatello, F., & Gavves, E. (2024). Mechanistic neural
networks for scientific machine learning. In Proceedings of the 41st International
Conference on Machine Learning (Vol. 235, pp. 40484–40501). Vienna, Austria:
ML Research Press.'
chicago: Pervez, Adeel A, Francesco Locatello, and Efstratios Gavves. “Mechanistic
Neural Networks for Scientific Machine Learning.” In Proceedings of the 41st
International Conference on Machine Learning, 235:40484–501. ML Research Press,
2024.
ieee: A. A. Pervez, F. Locatello, and E. Gavves, “Mechanistic neural networks for
scientific machine learning,” in Proceedings of the 41st International Conference
on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 40484–40501.
ista: 'Pervez AA, Locatello F, Gavves E. 2024. Mechanistic neural networks for scientific
machine learning. Proceedings of the 41st International Conference on Machine
Learning. ICML: International Conference on Machine Learning, PMLR, vol. 235,
40484–40501.'
mla: Pervez, Adeel A., et al. “Mechanistic Neural Networks for Scientific Machine
Learning.” Proceedings of the 41st International Conference on Machine Learning,
vol. 235, ML Research Press, 2024, pp. 40484–501.
short: A.A. Pervez, F. Locatello, E. Gavves, in:, Proceedings of the 41st International
Conference on Machine Learning, ML Research Press, 2024, pp. 40484–40501.
conference:
end_date: 2024-07-27
location: Vienna, Austria
name: 'ICML: International Conference on Machine Learning'
start_date: 2024-07-21
date_created: 2024-09-22T22:01:43Z
date_published: 2024-09-01T00:00:00Z
date_updated: 2026-06-18T17:59:46Z
day: '01'
ddc:
- '000'
department:
- _id: FrLo
external_id:
arxiv:
- '2402.13077'
intvolume: ' 235'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2402.13077
month: '09'
oa: 1
oa_version: Published Version
page: 40484-40501
publication: Proceedings of the 41st International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/alpz/mech-nn
scopus_import: '1'
status: public
title: Mechanistic neural networks for scientific machine learning
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 235
year: '2024'
...
---
_id: '14105'
abstract:
- lang: eng
text: "Despite their recent success, deep neural networks continue to perform poorly
when they encounter distribution shifts at test time. Many recently proposed approaches
try to counter this by aligning the model to the new distribution prior to inference.
With no labels available this requires unsupervised objectives to adapt the model
on the observed test data. In this paper, we propose Test-Time SelfTraining (TeST):
a technique that takes as input a model trained on some source data and a novel
data distribution at test time, and learns invariant and robust representations
using a student-teacher framework. We find that models adapted using TeST significantly
improve over baseline testtime adaptation algorithms. TeST achieves competitive
performance to modern domain adaptation algorithms [4, 43], while having access
to 5-10x less data at time of adaption. We thoroughly evaluate a variety of baselines
on two tasks:\r\nobject detection and image segmentation and find that models
adapted with TeST. We find that TeST sets the new stateof-the art for test-time
domain adaptation algorithms. "
article_processing_charge: No
arxiv: 1
author:
- first_name: Samarth
full_name: Sinha, Samarth
last_name: Sinha
- first_name: Peter
full_name: Gehler, Peter
last_name: Gehler
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Bernt
full_name: Schiele, Bernt
last_name: Schiele
citation:
ama: 'Sinha S, Gehler P, Locatello F, Schiele B. TeST: Test-time Self-Training under
distribution shift. In: 2023 IEEE/CVF Winter Conference on Applications of
Computer Vision. Institute of Electrical and Electronics Engineers; 2023.
doi:10.1109/wacv56688.2023.00278'
apa: 'Sinha, S., Gehler, P., Locatello, F., & Schiele, B. (2023). TeST: Test-time
Self-Training under distribution shift. In 2023 IEEE/CVF Winter Conference
on Applications of Computer Vision. Waikoloa, HI, United States: Institute
of Electrical and Electronics Engineers. https://doi.org/10.1109/wacv56688.2023.00278'
chicago: 'Sinha, Samarth, Peter Gehler, Francesco Locatello, and Bernt Schiele.
“TeST: Test-Time Self-Training under Distribution Shift.” In 2023 IEEE/CVF
Winter Conference on Applications of Computer Vision. Institute of Electrical
and Electronics Engineers, 2023. https://doi.org/10.1109/wacv56688.2023.00278.'
ieee: 'S. Sinha, P. Gehler, F. Locatello, and B. Schiele, “TeST: Test-time Self-Training
under distribution shift,” in 2023 IEEE/CVF Winter Conference on Applications
of Computer Vision, Waikoloa, HI, United States, 2023.'
ista: 'Sinha S, Gehler P, Locatello F, Schiele B. 2023. TeST: Test-time Self-Training
under distribution shift. 2023 IEEE/CVF Winter Conference on Applications of Computer
Vision. WACV: Winter Conference on Applications of Computer Vision.'
mla: 'Sinha, Samarth, et al. “TeST: Test-Time Self-Training under Distribution Shift.”
2023 IEEE/CVF Winter Conference on Applications of Computer Vision, Institute
of Electrical and Electronics Engineers, 2023, doi:10.1109/wacv56688.2023.00278.'
short: S. Sinha, P. Gehler, F. Locatello, B. Schiele, in:, 2023 IEEE/CVF Winter
Conference on Applications of Computer Vision, Institute of Electrical and Electronics
Engineers, 2023.
conference:
end_date: 2023-01-07
location: Waikoloa, HI, United States
name: 'WACV: Winter Conference on Applications of Computer Vision'
start_date: 2023-01-02
date_created: 2023-08-21T12:11:38Z
date_published: 2023-02-06T00:00:00Z
date_updated: 2023-09-06T10:26:56Z
day: '06'
department:
- _id: FrLo
doi: 10.1109/wacv56688.2023.00278
extern: '1'
external_id:
arxiv:
- '2209.11459'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2209.11459
month: '02'
oa: 1
oa_version: Preprint
publication: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision
publication_identifier:
eissn:
- 2642-9381
isbn:
- '9781665493475'
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'TeST: Test-time Self-Training under distribution shift'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14207'
abstract:
- lang: eng
text: The binding problem in human cognition, concerning how the brain represents
and connects objects within a fixed network of neural connections, remains a subject
of intense debate. Most machine learning efforts addressing this issue in an unsupervised
setting have focused on slot-based methods, which may be limiting due to their
discrete nature and difficulty to express uncertainty. Recently, the Complex AutoEncoder
was proposed as an alternative that learns continuous and distributed object-centric
representations. However, it is only applicable to simple toy data. In this paper,
we present Rotating Features, a generalization of complex-valued features to higher
dimensions, and a new evaluation procedure for extracting objects from distributed
representations. Additionally, we show the applicability of our approach to pre-trained
features. Together, these advancements enable us to scale distributed object-centric
representations from simple toy to real-world data. We believe this work advances
a new paradigm for addressing the binding problem in machine learning and has
the potential to inspire further innovation in the field.
article_number: '2306.00600'
article_processing_charge: No
arxiv: 1
author:
- first_name: Sindy
full_name: Löwe, Sindy
last_name: Löwe
- first_name: Phillip
full_name: Lippe, Phillip
last_name: Lippe
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Max
full_name: Welling, Max
last_name: Welling
citation:
ama: Löwe S, Lippe P, Locatello F, Welling M. Rotating features for object discovery.
arXiv. doi:10.48550/arXiv.2306.00600
apa: Löwe, S., Lippe, P., Locatello, F., & Welling, M. (n.d.). Rotating features
for object discovery. arXiv. https://doi.org/10.48550/arXiv.2306.00600
chicago: Löwe, Sindy, Phillip Lippe, Francesco Locatello, and Max Welling. “Rotating
Features for Object Discovery.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2306.00600.
ieee: S. Löwe, P. Lippe, F. Locatello, and M. Welling, “Rotating features for object
discovery,” arXiv. .
ista: Löwe S, Lippe P, Locatello F, Welling M. Rotating features for object discovery.
arXiv, 2306.00600.
mla: Löwe, Sindy, et al. “Rotating Features for Object Discovery.” ArXiv,
2306.00600, doi:10.48550/arXiv.2306.00600.
short: S. Löwe, P. Lippe, F. Locatello, M. Welling, ArXiv (n.d.).
corr_author: '1'
date_created: 2023-08-22T14:18:00Z
date_published: 2023-06-01T00:00:00Z
date_updated: 2024-10-09T21:06:53Z
day: '01'
department:
- _id: FrLo
doi: 10.48550/arXiv.2306.00600
external_id:
arxiv:
- '2306.00600'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2306.00600
month: '06'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: submitted
status: public
title: Rotating features for object discovery
type: preprint
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14208'
abstract:
- lang: eng
text: This paper focuses on over-parameterized deep neural networks (DNNs) with
ReLU activation functions and proves that when the data distribution is well-separated,
DNNs can achieve Bayes-optimal test error for classification while obtaining (nearly)
zero-training error under the lazy training regime. For this purpose, we unify
three interrelated concepts of overparameterization, benign overfitting, and the
Lipschitz constant of DNNs. Our results indicate that interpolating with smoother
functions leads to better generalization. Furthermore, we investigate the special
case where interpolating smooth ground-truth functions is performed by DNNs under
the Neural Tangent Kernel (NTK) regime for generalization. Our result demonstrates
that the generalization error converges to a constant order that only depends
on label noise and initialization noise, which theoretically verifies benign overfitting.
Our analysis provides a tight lower bound on the normalized margin under non-smooth
activation functions, as well as the minimum eigenvalue of NTK under high-dimensional
settings, which has its own interest in learning theory.
alternative_title:
- PMLR
article_processing_charge: No
arxiv: 1
author:
- first_name: Zhenyu
full_name: Zhu, Zhenyu
last_name: Zhu
- first_name: Fanghui
full_name: Liu, Fanghui
last_name: Liu
- first_name: Grigorios G
full_name: Chrysos, Grigorios G
last_name: Chrysos
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Volkan
full_name: Cevher, Volkan
last_name: Cevher
citation:
ama: 'Zhu Z, Liu F, Chrysos GG, Locatello F, Cevher V. Benign overfitting in deep
neural networks under lazy training. In: Proceedings of the 40th International
Conference on Machine Learning. Vol 202. ML Research Press; 2023:43105-43128.'
apa: 'Zhu, Z., Liu, F., Chrysos, G. G., Locatello, F., & Cevher, V. (2023).
Benign overfitting in deep neural networks under lazy training. In Proceedings
of the 40th International Conference on Machine Learning (Vol. 202, pp. 43105–43128).
Honolulu, Hawaii, United States: ML Research Press.'
chicago: Zhu, Zhenyu, Fanghui Liu, Grigorios G Chrysos, Francesco Locatello, and
Volkan Cevher. “Benign Overfitting in Deep Neural Networks under Lazy Training.”
In Proceedings of the 40th International Conference on Machine Learning,
202:43105–28. ML Research Press, 2023.
ieee: Z. Zhu, F. Liu, G. G. Chrysos, F. Locatello, and V. Cevher, “Benign overfitting
in deep neural networks under lazy training,” in Proceedings of the 40th International
Conference on Machine Learning, Honolulu, Hawaii, United States, 2023, vol.
202, pp. 43105–43128.
ista: Zhu Z, Liu F, Chrysos GG, Locatello F, Cevher V. 2023. Benign overfitting
in deep neural networks under lazy training. Proceedings of the 40th International
Conference on Machine Learning. International Conference on Machine Learning,
PMLR, vol. 202, 43105–43128.
mla: Zhu, Zhenyu, et al. “Benign Overfitting in Deep Neural Networks under Lazy
Training.” Proceedings of the 40th International Conference on Machine Learning,
vol. 202, ML Research Press, 2023, pp. 43105–28.
short: Z. Zhu, F. Liu, G.G. Chrysos, F. Locatello, V. Cevher, in:, Proceedings of
the 40th International Conference on Machine Learning, ML Research Press, 2023,
pp. 43105–43128.
conference:
end_date: 2023-07-29
location: Honolulu, Hawaii, United States
name: International Conference on Machine Learning
start_date: 2023-07-23
date_created: 2023-08-22T14:18:18Z
date_published: 2023-05-30T00:00:00Z
date_updated: 2023-09-13T08:46:46Z
day: '30'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2305.19377'
intvolume: ' 202'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2305.19377
month: '05'
oa: 1
oa_version: Preprint
page: 43105-43128
publication: Proceedings of the 40th International Conference on Machine Learning
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
status: public
title: Benign overfitting in deep neural networks under lazy training
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 202
year: '2023'
...
---
_id: '14209'
abstract:
- lang: eng
text: Diffusion models excel at generating photorealistic images from text-queries.
Naturally, many approaches have been proposed to use these generative abilities
to augment training datasets for downstream tasks, such as classification. However,
diffusion models are themselves trained on large noisily supervised, but nonetheless,
annotated datasets. It is an open question whether the generalization capabilities
of diffusion models beyond using the additional data of the pre-training process
for augmentation lead to improved downstream performance. We perform a systematic
evaluation of existing methods to generate images from diffusion models and study
new extensions to assess their benefit for data augmentation. While we find that
personalizing diffusion models towards the target data outperforms simpler prompting
strategies, we also show that using the training data of the diffusion model alone,
via a simple nearest neighbor retrieval procedure, leads to even stronger downstream
performance. Overall, our study probes the limitations of diffusion models for
data augmentation but also highlights its potential in generating new training
data to improve performance on simple downstream vision tasks.
article_number: '2304.10253'
article_processing_charge: No
arxiv: 1
author:
- first_name: Max F.
full_name: Burg, Max F.
last_name: Burg
- first_name: Florian
full_name: Wenzel, Florian
last_name: Wenzel
- first_name: Dominik
full_name: Zietlow, Dominik
last_name: Zietlow
- first_name: Max
full_name: Horn, Max
last_name: Horn
- first_name: Osama
full_name: Makansi, Osama
last_name: Makansi
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Chris
full_name: Russell, Chris
last_name: Russell
citation:
ama: Burg MF, Wenzel F, Zietlow D, et al. A data augmentation perspective on diffusion
models and retrieval. arXiv. doi:10.48550/arXiv.2304.10253
apa: Burg, M. F., Wenzel, F., Zietlow, D., Horn, M., Makansi, O., Locatello, F.,
& Russell, C. (n.d.). A data augmentation perspective on diffusion models
and retrieval. arXiv. https://doi.org/10.48550/arXiv.2304.10253
chicago: Burg, Max F., Florian Wenzel, Dominik Zietlow, Max Horn, Osama Makansi,
Francesco Locatello, and Chris Russell. “A Data Augmentation Perspective on Diffusion
Models and Retrieval.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2304.10253.
ieee: M. F. Burg et al., “A data augmentation perspective on diffusion models
and retrieval,” arXiv. .
ista: Burg MF, Wenzel F, Zietlow D, Horn M, Makansi O, Locatello F, Russell C. A
data augmentation perspective on diffusion models and retrieval. arXiv, 2304.10253.
mla: Burg, Max F., et al. “A Data Augmentation Perspective on Diffusion Models and
Retrieval.” ArXiv, 2304.10253, doi:10.48550/arXiv.2304.10253.
short: M.F. Burg, F. Wenzel, D. Zietlow, M. Horn, O. Makansi, F. Locatello, C. Russell,
ArXiv (n.d.).
date_created: 2023-08-22T14:18:43Z
date_published: 2023-04-20T00:00:00Z
date_updated: 2023-09-13T08:51:56Z
day: '20'
department:
- _id: FrLo
doi: 10.48550/arXiv.2304.10253
extern: '1'
external_id:
arxiv:
- '2304.10253'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2304.10253
month: '04'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: submitted
status: public
title: A data augmentation perspective on diffusion models and retrieval
type: preprint
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14210'
abstract:
- lang: eng
text: Recovering the latent factors of variation of high dimensional data has so
far focused on simple synthetic settings. Mostly building on unsupervised and
weakly-supervised objectives, prior work missed out on the positive implications
for representation learning on real world data. In this work, we propose to leverage
knowledge extracted from a diversified set of supervised tasks to learn a common
disentangled representation. Assuming each supervised task only depends on an
unknown subset of the factors of variation, we disentangle the feature space of
a supervised multi-task model, with features activating sparsely across different
tasks and information being shared as appropriate. Importantly, we never directly
observe the factors of variations but establish that access to multiple tasks
is sufficient for identifiability under sufficiency and minimality assumptions.
We validate our approach on six real world distribution shift benchmarks, and
different data modalities (images, text), demonstrating how disentangled representations
can be transferred to real settings.
article_number: '2304.07939'
article_processing_charge: No
arxiv: 1
author:
- first_name: Marco
full_name: Fumero, Marco
last_name: Fumero
- first_name: Florian
full_name: Wenzel, Florian
last_name: Wenzel
- first_name: Luca
full_name: Zancato, Luca
last_name: Zancato
- first_name: Alessandro
full_name: Achille, Alessandro
last_name: Achille
- first_name: Emanuele
full_name: Rodolà, Emanuele
last_name: Rodolà
- first_name: Stefano
full_name: Soatto, Stefano
last_name: Soatto
- first_name: Bernhard
full_name: Schölkopf, Bernhard
last_name: Schölkopf
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: Fumero M, Wenzel F, Zancato L, et al. Leveraging sparse and shared feature
activations for disentangled representation learning. arXiv. doi:10.48550/arXiv.2304.07939
apa: Fumero, M., Wenzel, F., Zancato, L., Achille, A., Rodolà, E., Soatto, S., …
Locatello, F. (n.d.). Leveraging sparse and shared feature activations for disentangled
representation learning. arXiv. https://doi.org/10.48550/arXiv.2304.07939
chicago: Fumero, Marco, Florian Wenzel, Luca Zancato, Alessandro Achille, Emanuele
Rodolà, Stefano Soatto, Bernhard Schölkopf, and Francesco Locatello. “Leveraging
Sparse and Shared Feature Activations for Disentangled Representation Learning.”
ArXiv, n.d. https://doi.org/10.48550/arXiv.2304.07939.
ieee: M. Fumero et al., “Leveraging sparse and shared feature activations
for disentangled representation learning,” arXiv. .
ista: Fumero M, Wenzel F, Zancato L, Achille A, Rodolà E, Soatto S, Schölkopf B,
Locatello F. Leveraging sparse and shared feature activations for disentangled
representation learning. arXiv, 2304.07939.
mla: Fumero, Marco, et al. “Leveraging Sparse and Shared Feature Activations for
Disentangled Representation Learning.” ArXiv, 2304.07939, doi:10.48550/arXiv.2304.07939.
short: M. Fumero, F. Wenzel, L. Zancato, A. Achille, E. Rodolà, S. Soatto, B. Schölkopf,
F. Locatello, ArXiv (n.d.).
corr_author: '1'
date_created: 2023-08-22T14:19:03Z
date_published: 2023-04-17T00:00:00Z
date_updated: 2024-10-09T21:06:54Z
day: '17'
department:
- _id: FrLo
doi: 10.48550/arXiv.2304.07939
external_id:
arxiv:
- '2304.07939'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2304.07939
month: '04'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: submitted
status: public
title: Leveraging sparse and shared feature activations for disentangled representation
learning
type: preprint
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14211'
abstract:
- lang: eng
text: 'Causal discovery methods are intrinsically constrained by the set of assumptions
needed to ensure structure identifiability. Moreover additional restrictions are
often imposed in order to simplify the inference task: this is the case for the
Gaussian noise assumption on additive non-linear models, which is common to many
causal discovery approaches. In this paper we show the shortcomings of inference
under this hypothesis, analyzing the risk of edge inversion under violation of
Gaussianity of the noise terms. Then, we propose a novel method for inferring
the topological ordering of the variables in the causal graph, from data generated
according to an additive non-linear model with a generic noise distribution. This
leads to NoGAM (Not only Gaussian Additive noise Models), a causal discovery algorithm
with a minimal set of assumptions and state of the art performance, experimentally
benchmarked on synthetic data.'
article_processing_charge: No
arxiv: 1
author:
- first_name: Francesco
full_name: Montagna, Francesco
last_name: Montagna
- first_name: Nicoletta
full_name: Noceti, Nicoletta
last_name: Noceti
- first_name: Lorenzo
full_name: Rosasco, Lorenzo
last_name: Rosasco
- first_name: Kun
full_name: Zhang, Kun
last_name: Zhang
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Montagna F, Noceti N, Rosasco L, Zhang K, Locatello F. Causal discovery with
score matching on additive models with arbitrary noise. In: 2nd Conference
on Causal Learning and Reasoning. ; 2023.'
apa: Montagna, F., Noceti, N., Rosasco, L., Zhang, K., & Locatello, F. (2023).
Causal discovery with score matching on additive models with arbitrary noise.
In 2nd Conference on Causal Learning and Reasoning. Tübingen, Germany.
chicago: Montagna, Francesco, Nicoletta Noceti, Lorenzo Rosasco, Kun Zhang, and
Francesco Locatello. “Causal Discovery with Score Matching on Additive Models
with Arbitrary Noise.” In 2nd Conference on Causal Learning and Reasoning,
2023.
ieee: F. Montagna, N. Noceti, L. Rosasco, K. Zhang, and F. Locatello, “Causal discovery
with score matching on additive models with arbitrary noise,” in 2nd Conference
on Causal Learning and Reasoning, Tübingen, Germany, 2023.
ista: 'Montagna F, Noceti N, Rosasco L, Zhang K, Locatello F. 2023. Causal discovery
with score matching on additive models with arbitrary noise. 2nd Conference on
Causal Learning and Reasoning. CLeaR: Conference on Causal Learning and Reasoning.'
mla: Montagna, Francesco, et al. “Causal Discovery with Score Matching on Additive
Models with Arbitrary Noise.” 2nd Conference on Causal Learning and Reasoning,
2023.
short: F. Montagna, N. Noceti, L. Rosasco, K. Zhang, F. Locatello, in:, 2nd Conference
on Causal Learning and Reasoning, 2023.
conference:
end_date: 2023-04-14
location: Tübingen, Germany
name: 'CLeaR: Conference on Causal Learning and Reasoning'
start_date: 2023-04-11
date_created: 2023-08-22T14:19:21Z
date_published: 2023-04-01T00:00:00Z
date_updated: 2024-10-14T12:30:04Z
day: '01'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2304.03265'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2304.03265
month: '04'
oa: 1
oa_version: Preprint
publication: 2nd Conference on Causal Learning and Reasoning
publication_status: published
quality_controlled: '1'
scopus_import: '1'
status: public
title: Causal discovery with score matching on additive models with arbitrary noise
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14212'
abstract:
- lang: eng
text: This paper demonstrates how to discover the whole causal graph from the second
derivative of the log-likelihood in observational non-linear additive Gaussian
noise models. Leveraging scalable machine learning approaches to approximate the
score function ∇logp(X), we extend the work of Rolland et al. (2022) that only
recovers the topological order from the score and requires an expensive pruning
step removing spurious edges among those admitted by the ordering. Our analysis
leads to DAS (acronym for Discovery At Scale), a practical algorithm that reduces
the complexity of the pruning by a factor proportional to the graph size. In practice,
DAS achieves competitive accuracy with current state-of-the-art while being over
an order of magnitude faster. Overall, our approach enables principled and scalable
causal discovery, significantly lowering the compute bar.
article_processing_charge: No
arxiv: 1
author:
- first_name: Francesco
full_name: Montagna, Francesco
last_name: Montagna
- first_name: Nicoletta
full_name: Noceti, Nicoletta
last_name: Noceti
- first_name: Lorenzo
full_name: Rosasco, Lorenzo
last_name: Rosasco
- first_name: Kun
full_name: Zhang, Kun
last_name: Zhang
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Montagna F, Noceti N, Rosasco L, Zhang K, Locatello F. Scalable causal discovery
with score matching. In: 2nd Conference on Causal Learning and Reasoning.
; 2023.'
apa: Montagna, F., Noceti, N., Rosasco, L., Zhang, K., & Locatello, F. (2023).
Scalable causal discovery with score matching. In 2nd Conference on Causal
Learning and Reasoning. Tübingen, Germany.
chicago: Montagna, Francesco, Nicoletta Noceti, Lorenzo Rosasco, Kun Zhang, and
Francesco Locatello. “Scalable Causal Discovery with Score Matching.” In 2nd
Conference on Causal Learning and Reasoning, 2023.
ieee: F. Montagna, N. Noceti, L. Rosasco, K. Zhang, and F. Locatello, “Scalable
causal discovery with score matching,” in 2nd Conference on Causal Learning
and Reasoning, Tübingen, Germany, 2023.
ista: 'Montagna F, Noceti N, Rosasco L, Zhang K, Locatello F. 2023. Scalable causal
discovery with score matching. 2nd Conference on Causal Learning and Reasoning.
CLeaR: Conference on Causal Learning and Reasoning.'
mla: Montagna, Francesco, et al. “Scalable Causal Discovery with Score Matching.”
2nd Conference on Causal Learning and Reasoning, 2023.
short: F. Montagna, N. Noceti, L. Rosasco, K. Zhang, F. Locatello, in:, 2nd Conference
on Causal Learning and Reasoning, 2023.
conference:
end_date: 2023-04-14
location: Tübingen, Germany
name: 'CLeaR: Conference on Causal Learning and Reasoning'
start_date: 2023-04-11
date_created: 2023-08-22T14:19:40Z
date_published: 2023-04-01T00:00:00Z
date_updated: 2024-10-14T12:30:15Z
day: '01'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2304.03382'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2304.03382
month: '04'
oa: 1
oa_version: Preprint
publication: 2nd Conference on Causal Learning and Reasoning
publication_status: published
quality_controlled: '1'
scopus_import: '1'
status: public
title: Scalable causal discovery with score matching
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14214'
abstract:
- lang: eng
text: 'Recent years have seen a surge of interest in learning high-level causal
representations from low-level image pairs under interventions. Yet, existing
efforts are largely limited to simple synthetic settings that are far away from
real-world problems. In this paper, we present Causal Triplet, a causal representation
learning benchmark featuring not only visually more complex scenes, but also two
crucial desiderata commonly overlooked in previous works: (i) an actionable counterfactual
setting, where only certain object-level variables allow for counterfactual observations
whereas others do not; (ii) an interventional downstream task with an emphasis
on out-of-distribution robustness from the independent causal mechanisms principle.
Through extensive experiments, we find that models built with the knowledge of
disentangled or object-centric representations significantly outperform their
distributed counterparts. However, recent causal representation learning methods
still struggle to identify such latent structures, indicating substantial challenges
and opportunities for future work.'
article_processing_charge: No
arxiv: 1
author:
- first_name: Yuejiang
full_name: Liu, Yuejiang
last_name: Liu
- first_name: Alexandre
full_name: Alahi, Alexandre
last_name: Alahi
- first_name: Chris
full_name: Russell, Chris
last_name: Russell
- first_name: Max
full_name: Horn, Max
last_name: Horn
- first_name: Dominik
full_name: Zietlow, Dominik
last_name: Zietlow
- first_name: Bernhard
full_name: Schölkopf, Bernhard
last_name: Schölkopf
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Liu Y, Alahi A, Russell C, et al. Causal triplet: An open challenge for intervention-centric
causal representation learning. In: 2nd Conference on Causal Learning and Reasoning.
; 2023.'
apa: 'Liu, Y., Alahi, A., Russell, C., Horn, M., Zietlow, D., Schölkopf, B., &
Locatello, F. (2023). Causal triplet: An open challenge for intervention-centric
causal representation learning. In 2nd Conference on Causal Learning and Reasoning.
Tübingen, Germany.'
chicago: 'Liu, Yuejiang, Alexandre Alahi, Chris Russell, Max Horn, Dominik Zietlow,
Bernhard Schölkopf, and Francesco Locatello. “Causal Triplet: An Open Challenge
for Intervention-Centric Causal Representation Learning.” In 2nd Conference
on Causal Learning and Reasoning, 2023.'
ieee: 'Y. Liu et al., “Causal triplet: An open challenge for intervention-centric
causal representation learning,” in 2nd Conference on Causal Learning and Reasoning,
Tübingen, Germany, 2023.'
ista: 'Liu Y, Alahi A, Russell C, Horn M, Zietlow D, Schölkopf B, Locatello F. 2023.
Causal triplet: An open challenge for intervention-centric causal representation
learning. 2nd Conference on Causal Learning and Reasoning. CLeaR: Conference on
Causal Learning and Reasoning.'
mla: 'Liu, Yuejiang, et al. “Causal Triplet: An Open Challenge for Intervention-Centric
Causal Representation Learning.” 2nd Conference on Causal Learning and Reasoning,
2023.'
short: Y. Liu, A. Alahi, C. Russell, M. Horn, D. Zietlow, B. Schölkopf, F. Locatello,
in:, 2nd Conference on Causal Learning and Reasoning, 2023.
conference:
end_date: 2023-04-14
location: Tübingen, Germany
name: 'CLeaR: Conference on Causal Learning and Reasoning'
start_date: 2023-04-11
date_created: 2023-08-22T14:20:18Z
date_published: 2023-04-12T00:00:00Z
date_updated: 2024-10-14T12:30:42Z
day: '12'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2301.05169'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2301.05169
month: '04'
oa: 1
oa_version: Preprint
publication: 2nd Conference on Causal Learning and Reasoning
publication_status: published
quality_controlled: '1'
status: public
title: 'Causal triplet: An open challenge for intervention-centric causal representation
learning'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
OA_type: green
_id: '14216'
abstract:
- lang: eng
text: CLIP proved that aligning visual and language spaces is key to solving many
vision tasks without explicit training, but required to train image and text encoders
from scratch on a huge dataset. LiT improved this by only training the text encoder
and using a pre-trained vision network. In this paper, we show that a common space
can be created without any training at all, using single-domain encoders (trained
with or without supervision) and a much smaller amount of image-text pairs. Furthermore,
our model has unique properties. Most notably, deploying a new version with updated
training samples can be done in a matter of seconds. Additionally, the representations
in the common space are easily interpretable as every dimension corresponds to
the similarity of the input to a unique entry in the multimodal dataset. Experiments
on standard zero-shot visual benchmarks demonstrate the typical transfer ability
of image-text models. Overall, our method represents a simple yet surprisingly
strong baseline for foundation multi-modal models, raising important questions
on their data efficiency and on the role of retrieval in machine learning.
acknowledgement: "AN, MF, and FL partially worked on ASIF when they were at Amazon
Web Services in Tübingen,\r\nGermany. This paper is financially supported by the
PRIN 2020 project no.2020TA3K9N (LEGO.AI), PNRR MUR project PE0000013-FAIR, and
ERC Grant no.802554 (SPECGEO)."
alternative_title:
- Advances in Neural Information Processing Systems
article_processing_charge: No
arxiv: 1
author:
- first_name: Antonio
full_name: Norelli, Antonio
last_name: Norelli
- first_name: Marco
full_name: Fumero, Marco
last_name: Fumero
- first_name: Valentino
full_name: Maiorca, Valentino
last_name: Maiorca
- first_name: Luca
full_name: Moschella, Luca
last_name: Moschella
- first_name: Emanuele
full_name: Rodolà, Emanuele
last_name: Rodolà
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Norelli A, Fumero M, Maiorca V, Moschella L, Rodolà E, Locatello F. ASIF:
Coupled data turns unimodal models to multimodal without training. In: 37th
Conference on Neural Information Processing Systems. Vol 36. Neural Information
Processing Systems Foundation; 2023:15303-15319.'
apa: 'Norelli, A., Fumero, M., Maiorca, V., Moschella, L., Rodolà, E., & Locatello,
F. (2023). ASIF: Coupled data turns unimodal models to multimodal without training.
In 37th Conference on Neural Information Processing Systems (Vol. 36, pp.
15303–15319). New Orleans, LA, United States: Neural Information Processing Systems
Foundation.'
chicago: 'Norelli, Antonio, Marco Fumero, Valentino Maiorca, Luca Moschella, Emanuele
Rodolà, and Francesco Locatello. “ASIF: Coupled Data Turns Unimodal Models to
Multimodal without Training.” In 37th Conference on Neural Information Processing
Systems, 36:15303–19. Neural Information Processing Systems Foundation, 2023.'
ieee: 'A. Norelli, M. Fumero, V. Maiorca, L. Moschella, E. Rodolà, and F. Locatello,
“ASIF: Coupled data turns unimodal models to multimodal without training,” in
37th Conference on Neural Information Processing Systems, New Orleans,
LA, United States, 2023, vol. 36, pp. 15303–15319.'
ista: 'Norelli A, Fumero M, Maiorca V, Moschella L, Rodolà E, Locatello F. 2023.
ASIF: Coupled data turns unimodal models to multimodal without training. 37th
Conference on Neural Information Processing Systems. NeurIPS: Neural Information
Processing Systems, Advances in Neural Information Processing Systems, vol. 36,
15303–15319.'
mla: 'Norelli, Antonio, et al. “ASIF: Coupled Data Turns Unimodal Models to Multimodal
without Training.” 37th Conference on Neural Information Processing Systems,
vol. 36, Neural Information Processing Systems Foundation, 2023, pp. 15303–19.'
short: A. Norelli, M. Fumero, V. Maiorca, L. Moschella, E. Rodolà, F. Locatello,
in:, 37th Conference on Neural Information Processing Systems, Neural Information
Processing Systems Foundation, 2023, pp. 15303–15319.
conference:
end_date: 2023-12-14
location: New Orleans, LA, United States
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2023-12-12
corr_author: '1'
date_created: 2023-08-22T14:22:04Z
date_published: 2023-10-04T00:00:00Z
date_updated: 2025-05-14T11:28:52Z
day: '04'
ddc:
- '000'
department:
- _id: FrLo
external_id:
arxiv:
- '2210.01738'
file:
- access_level: open_access
checksum: e51c90300b92d7135050da5c9e3a8015
content_type: application/pdf
creator: dernst
date_created: 2025-02-04T12:16:13Z
date_updated: 2025-02-04T12:16:13Z
file_id: '18994'
file_name: 2023_NeurIPS_Fumero.pdf
file_size: 12648978
relation: main_file
success: 1
file_date_updated: 2025-02-04T12:16:13Z
has_accepted_license: '1'
intvolume: ' 36'
language:
- iso: eng
month: '10'
oa: 1
oa_version: Preprint
page: 15303-15319
publication: 37th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9781713899921'
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
related_material:
link:
- relation: software
url: https://github.com/noranta4/ASIF
status: public
title: 'ASIF: Coupled data turns unimodal models to multimodal without training'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 36
year: '2023'
...
---
_id: '14217'
abstract:
- lang: eng
text: 'Neural networks embed the geometric structure of a data manifold lying in
a high-dimensional space into latent representations. Ideally, the distribution
of the data points in the latent space should depend only on the task, the data,
the loss, and other architecture-specific constraints. However, factors such as
the random weights initialization, training hyperparameters, or other sources
of randomness in the training phase may induce incoherent latent spaces that hinder
any form of reuse. Nevertheless, we empirically observe that, under the same data
and modeling choices, the angles between the encodings within distinct latent
spaces do not change. In this work, we propose the latent similarity between each
sample and a fixed set of anchors as an alternative data representation, demonstrating
that it can enforce the desired invariances without any additional training. We
show how neural architectures can leverage these relative representations to guarantee,
in practice, invariance to latent isometries and rescalings, effectively enabling
latent space communication: from zero-shot model stitching to latent space comparison
between diverse settings. We extensively validate the generalization capability
of our approach on different datasets, spanning various modalities (images, text,
graphs), tasks (e.g., classification, reconstruction) and architectures (e.g.,
CNNs, GCNs, transformers).'
article_processing_charge: No
arxiv: 1
author:
- first_name: Luca
full_name: Moschella, Luca
last_name: Moschella
- first_name: Valentino
full_name: Maiorca, Valentino
last_name: Maiorca
- first_name: Marco
full_name: Fumero, Marco
last_name: Fumero
- first_name: Antonio
full_name: Norelli, Antonio
last_name: Norelli
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
- first_name: Emanuele
full_name: Rodolà, Emanuele
last_name: Rodolà
citation:
ama: 'Moschella L, Maiorca V, Fumero M, Norelli A, Locatello F, Rodolà E. Relative
representations enable zero-shot latent space communication. In: The 11th International
Conference on Learning Representations. ; 2023.'
apa: Moschella, L., Maiorca, V., Fumero, M., Norelli, A., Locatello, F., & Rodolà,
E. (2023). Relative representations enable zero-shot latent space communication.
In The 11th International Conference on Learning Representations. Kigali,
Rwanda.
chicago: Moschella, Luca, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco
Locatello, and Emanuele Rodolà. “Relative Representations Enable Zero-Shot Latent
Space Communication.” In The 11th International Conference on Learning Representations,
2023.
ieee: L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, and E. Rodolà,
“Relative representations enable zero-shot latent space communication,” in The
11th International Conference on Learning Representations, Kigali, Rwanda,
2023.
ista: Moschella L, Maiorca V, Fumero M, Norelli A, Locatello F, Rodolà E. 2023.
Relative representations enable zero-shot latent space communication. The 11th
International Conference on Learning Representations. International Conference
on Machine Learning Representations.
mla: Moschella, Luca, et al. “Relative Representations Enable Zero-Shot Latent Space
Communication.” The 11th International Conference on Learning Representations,
2023.
short: L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, E. Rodolà,
in:, The 11th International Conference on Learning Representations, 2023.
conference:
end_date: 2023-05-05
location: Kigali, Rwanda
name: International Conference on Machine Learning Representations
start_date: 2023-05-01
date_created: 2023-08-22T14:22:20Z
date_published: 2023-05-01T00:00:00Z
date_updated: 2023-09-13T09:44:26Z
day: '01'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2209.15430'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2209.15430
month: '05'
oa: 1
oa_version: Preprint
publication: The 11th International Conference on Learning Representations
publication_status: published
quality_controlled: '1'
status: public
title: Relative representations enable zero-shot latent space communication
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '14218'
abstract:
- lang: eng
text: Humans naturally decompose their environment into entities at the appropriate
level of abstraction to act in the world. Allowing machine learning algorithms
to derive this decomposition in an unsupervised way has become an important line
of research. However, current methods are restricted to simulated data or require
additional information in the form of motion or depth in order to successfully
discover objects. In this work, we overcome this limitation by showing that reconstructing
features from models trained in a self-supervised manner is a sufficient training
signal for object-centric representations to arise in a fully unsupervised way.
Our approach, DINOSAUR, significantly out-performs existing image-based object-centric
learning models on simulated data and is the first unsupervised object-centric
model that scales to real-world datasets such as COCO and PASCAL VOC. DINOSAUR
is conceptually simple and shows competitive performance compared to more involved
pipelines from the computer vision literature.
article_processing_charge: No
arxiv: 1
author:
- first_name: Maximilian
full_name: Seitzer, Maximilian
last_name: Seitzer
- first_name: Max
full_name: Horn, Max
last_name: Horn
- first_name: Andrii
full_name: Zadaianchuk, Andrii
last_name: Zadaianchuk
- first_name: Dominik
full_name: Zietlow, Dominik
last_name: Zietlow
- first_name: Tianjun
full_name: Xiao, Tianjun
last_name: Xiao
- first_name: Carl-Johann Simon-Gabriel
full_name: Carl-Johann Simon-Gabriel, Carl-Johann Simon-Gabriel
last_name: Carl-Johann Simon-Gabriel
- first_name: Tong
full_name: He, Tong
last_name: He
- first_name: Zheng
full_name: Zhang, Zheng
last_name: Zhang
- first_name: Bernhard
full_name: Schölkopf, Bernhard
last_name: Schölkopf
- first_name: Thomas
full_name: Brox, Thomas
last_name: Brox
- first_name: Francesco
full_name: Locatello, Francesco
id: 26cfd52f-2483-11ee-8040-88983bcc06d4
last_name: Locatello
orcid: 0000-0002-4850-0683
citation:
ama: 'Seitzer M, Horn M, Zadaianchuk A, et al. Bridging the gap to real-world object-centric
learning. In: The 11th International Conference on Learning Representations.
; 2023.'
apa: Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Carl-Johann
Simon-Gabriel, C.-J. S.-G., … Locatello, F. (2023). Bridging the gap to real-world
object-centric learning. In The 11th International Conference on Learning Representations.
Kigali, Rwanda.
chicago: Seitzer, Maximilian, Max Horn, Andrii Zadaianchuk, Dominik Zietlow, Tianjun
Xiao, Carl-Johann Simon-Gabriel Carl-Johann Simon-Gabriel, Tong He, et al. “Bridging
the Gap to Real-World Object-Centric Learning.” In The 11th International Conference
on Learning Representations, 2023.
ieee: M. Seitzer et al., “Bridging the gap to real-world object-centric learning,”
in The 11th International Conference on Learning Representations, Kigali,
Rwanda, 2023.
ista: 'Seitzer M, Horn M, Zadaianchuk A, Zietlow D, Xiao T, Carl-Johann Simon-Gabriel
C-JS-G, He T, Zhang Z, Schölkopf B, Brox T, Locatello F. 2023. Bridging the gap
to real-world object-centric learning. The 11th International Conference on Learning
Representations. ICLR: International Conference on Learning Representations.'
mla: Seitzer, Maximilian, et al. “Bridging the Gap to Real-World Object-Centric
Learning.” The 11th International Conference on Learning Representations,
2023.
short: M. Seitzer, M. Horn, A. Zadaianchuk, D. Zietlow, T. Xiao, C.-J.S.-G. Carl-Johann
Simon-Gabriel, T. He, Z. Zhang, B. Schölkopf, T. Brox, F. Locatello, in:, The
11th International Conference on Learning Representations, 2023.
conference:
end_date: 2023-05-05
location: Kigali, Rwanda
name: 'ICLR: International Conference on Learning Representations'
start_date: 2023-05-01
date_created: 2023-08-22T14:22:41Z
date_published: 2023-05-10T00:00:00Z
date_updated: 2024-10-14T12:30:54Z
day: '10'
department:
- _id: FrLo
extern: '1'
external_id:
arxiv:
- '2209.14860'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2209.14860
month: '05'
oa: 1
oa_version: Preprint
publication: The 11th International Conference on Learning Representations
publication_status: published
quality_controlled: '1'
status: public
title: Bridging the gap to real-world object-centric learning
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...