{"file":[{"file_name":"2025_PMLR_Montagna.pdf","success":1,"checksum":"968c471bb1f682cf823b2d4cadea8a3f","creator":"dernst","relation":"main_file","file_size":1030280,"date_updated":"2026-01-05T09:51:28Z","date_created":"2026-01-05T09:51:28Z","access_level":"open_access","file_id":"20939","content_type":"application/pdf"}],"date_updated":"2026-01-05T09:54:59Z","has_accepted_license":"1","day":"18","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","external_id":{"arxiv":["2405.16924"]},"publication":"Transactions on Machine Learning Research","corr_author":"1","year":"2025","OA_place":"publisher","month":"12","author":[{"full_name":"Montagna, Francesco","first_name":"Francesco","last_name":"Montagna","id":"353afc8e-19f4-11f0-9db9-811f1723c83f"},{"last_name":"Cairney-Leeming","full_name":"Cairney-Leeming, Maximilian T","first_name":"Maximilian T","id":"2214a80c-31f8-11ee-a48d-cf52cc58759b"},{"last_name":"Sridhar","full_name":"Sridhar, Dhanya","first_name":"Dhanya"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco","last_name":"Locatello"}],"file_date_updated":"2026-01-05T09:51:28Z","citation":{"ieee":"F. Montagna, M. T. Cairney-Leeming, D. Sridhar, and F. Locatello, “Demystifying amortized causal discovery with transformers,” <i>Transactions on Machine Learning Research</i>. ML Research Press, 2025.","mla":"Montagna, Francesco, et al. “Demystifying Amortized Causal Discovery with Transformers.” <i>Transactions on Machine Learning Research</i>, ML Research Press, 2025.","chicago":"Montagna, Francesco, Maximilian T Cairney-Leeming, Dhanya Sridhar, and Francesco Locatello. “Demystifying Amortized Causal Discovery with Transformers.” <i>Transactions on Machine Learning Research</i>. ML Research Press, 2025.","apa":"Montagna, F., Cairney-Leeming, M. T., Sridhar, D., &#38; Locatello, F. (2025). Demystifying amortized causal discovery with transformers. <i>Transactions on Machine Learning Research</i>. ML Research Press.","ama":"Montagna F, Cairney-Leeming MT, Sridhar D, Locatello F. Demystifying amortized causal discovery with transformers. <i>Transactions on Machine Learning Research</i>. 2025.","short":"F. Montagna, M.T. Cairney-Leeming, D. Sridhar, F. Locatello, Transactions on Machine Learning Research (2025).","ista":"Montagna F, Cairney-Leeming MT, Sridhar D, Locatello F. 2025. Demystifying amortized causal discovery with transformers. Transactions on Machine Learning Research."},"type":"journal_article","alternative_title":["TMLR"],"date_published":"2025-12-18T00:00:00Z","oa_version":"Published Version","tmp":{"short":"CC BY (4.0)","image":"/images/cc_by.png","legal_code_url":"https://creativecommons.org/licenses/by/4.0/legalcode","name":"Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)"},"department":[{"_id":"FrLo"}],"OA_type":"gold","ddc":["000"],"status":"public","article_processing_charge":"No","publication_identifier":{"eissn":["2835-8856"]},"quality_controlled":"1","PlanS_conform":"1","abstract":[{"lang":"eng","text":" Supervised learning for causal discovery from observational data often achieves competitive performance despite seemingly avoiding the explicit assumptions that traditional methods require for identifiability. In this work, we analyze CSIvA (Ke et al., 2023) on bivariate causal models, a transformer architecture for amortized inference promising to train on synthetic data and transfer to real ones. First, we bridge the gap with identifiability theory, showing that the training distribution implicitly defines a prior on the causal model of the test observations: consistent with classical approaches, good performance is achieved when we have a good prior on the test data, and the underlying model is identifiable. Second, we find that CSIvA can not generalize to classes of causal models unseen during training: to overcome this limitation, we theoretically and empirically analyze \\textit{when} training CSIvA on datasets generated by multiple identifiable causal models with different structural assumptions improves its generalization at test time. Overall, we find that amortized causal discovery still adheres to identifiability theory, violating the previous hypothesis from Lopez-Paz et al. (2015) that supervised learning methods could overcome its restrictions."}],"title":"Demystifying amortized causal discovery with transformers","_id":"20934","language":[{"iso":"eng"}],"related_material":{"link":[{"url":"https://github.com/francescomontagna/learning-to-induce.git","relation":"software"}]},"article_type":"original","scopus_import":"1","publication_status":"published","publisher":"ML Research Press","oa":1,"date_created":"2026-01-04T23:01:35Z","arxiv":1}