{"publication":"arXiv","acknowledgement":"We thank Piersilvio De Bartolomeis, and the full Causal Learning and Artificial Intelligence (CLAI) group at ISTA for the extremely helpful discussions. Riccardo Cadei was supported by a Google Research Scholar Award and a Google Initiated Gift to Francesco Locatello. We thank the Social Immunity team at ISTA particularly Michaela Hönigsberger and Wilfrid Jean Louis, for supporting the ecological experiment and Farnaz Beikzadeh Abbasi, Luisa Fiebig and Martin Estermann for annotating ant behavior in ISTAnt.","abstract":[{"text":"Machine Learning and AI have the potential to transform data-driven\r\nscientific discovery, enabling accurate predictions for several scientific\r\nphenomena. As many scientific questions are inherently causal, this paper looks\r\nat the causal inference task of treatment effect estimation, where the outcome\r\nof interest is recorded in high-dimensional observations in a Randomized\r\nControlled Trial (RCT). Despite being the simplest possible causal setting and\r\na perfect fit for deep learning, we theoretically find that many common choices\r\nin the literature may lead to biased estimates. To test the practical impact of\r\nthese considerations, we recorded ISTAnt, the first real-world benchmark for\r\ncausal inference downstream tasks on high-dimensional observations as an RCT\r\nstudying how garden ants (Lasius neglectus) respond to microparticles applied\r\nonto their colony members by hygienic grooming. Comparing 6 480 models\r\nfine-tuned from state-of-the-art visual backbones, we find that the sampling\r\nand modeling choices significantly affect the accuracy of the causal estimate,\r\nand that classification accuracy is not a proxy thereof. We further validated\r\nthe analysis, repeating it on a synthetically generated visual data set\r\ncontrolling the causal model. Our results suggest that future benchmarks should\r\ncarefully consider real downstream scientific questions, especially causal\r\nones. Further, we highlight guidelines for representation learning methods to\r\nhelp answer causal questions in the sciences.","lang":"eng"}],"date_updated":"2025-01-14T07:34:05Z","publication_status":"submitted","corr_author":"1","doi":"10.48550/arXiv.2405.17151","OA_place":"repository","external_id":{"arxiv":["2405.17151"]},"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2405.17151"}],"department":[{"_id":"SyCr"},{"_id":"FrLo"},{"_id":"GradSch"}],"_id":"18847","title":"Smoke and mirrors in causal downstream tasks","citation":{"mla":"Cadei, Riccardo, et al. “Smoke and Mirrors in Causal Downstream Tasks.” <i>ArXiv</i>, 2405.17151, doi:<a href=\"https://doi.org/10.48550/arXiv.2405.17151\">10.48550/arXiv.2405.17151</a>.","ieee":"R. Cadei, L. Lindorfer, S. Cremer, C. Schmid, and F. Locatello, “Smoke and mirrors in causal downstream tasks,” <i>arXiv</i>. .","ista":"Cadei R, Lindorfer L, Cremer S, Schmid C, Locatello F. Smoke and mirrors in causal downstream tasks. arXiv, 2405.17151.","chicago":"Cadei, Riccardo, Lukas Lindorfer, Sylvia Cremer, Cordelia Schmid, and Francesco Locatello. “Smoke and Mirrors in Causal Downstream Tasks.” <i>ArXiv</i>, n.d. <a href=\"https://doi.org/10.48550/arXiv.2405.17151\">https://doi.org/10.48550/arXiv.2405.17151</a>.","short":"R. Cadei, L. Lindorfer, S. Cremer, C. Schmid, F. Locatello, ArXiv (n.d.).","ama":"Cadei R, Lindorfer L, Cremer S, Schmid C, Locatello F. Smoke and mirrors in causal downstream tasks. <i>arXiv</i>. doi:<a href=\"https://doi.org/10.48550/arXiv.2405.17151\">10.48550/arXiv.2405.17151</a>","apa":"Cadei, R., Lindorfer, L., Cremer, S., Schmid, C., &#38; Locatello, F. (n.d.). Smoke and mirrors in causal downstream tasks. <i>arXiv</i>. <a href=\"https://doi.org/10.48550/arXiv.2405.17151\">https://doi.org/10.48550/arXiv.2405.17151</a>"},"oa_version":"Preprint","day":"27","year":"2024","language":[{"iso":"eng"}],"author":[{"id":"0fa8b76f-72f0-11ef-b75a-a5da96e5ad6b","last_name":"Cadei","first_name":"Riccardo","full_name":"Cadei, Riccardo"},{"last_name":"Lindorfer","first_name":"Lukas","id":"85f0e6d3-06b3-11ec-8982-8c5049fa4455","full_name":"Lindorfer, Lukas"},{"orcid":"0000-0002-2193-3868","full_name":"Cremer, Sylvia","last_name":"Cremer","first_name":"Sylvia","id":"2F64EC8C-F248-11E8-B48F-1D18A9856A87"},{"last_name":"Schmid","first_name":"Cordelia","full_name":"Schmid, Cordelia"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","first_name":"Francesco","last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683"}],"article_number":"2405.17151","oa":1,"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","status":"public","date_published":"2024-05-27T00:00:00Z","month":"05","date_created":"2025-01-14T07:27:26Z","type":"preprint","arxiv":1,"article_processing_charge":"No"}