Self-compatibility: Evaluating causal discovery without ground truth

Faller PM, Vankadara LC, Mastakouri AA, Locatello F, Janzing D. Self-compatibility: Evaluating causal discovery without ground truth. arXiv, 2307.09552.

Download (ext.)

Preprint | Submitted | English
Author
Faller, Philipp M.; Vankadara, Leena Chennuru; Mastakouri, Atalanti A.; Locatello, FrancescoISTA ; Janzing, Dominik
Department
Abstract
As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data. This is concerning, given that simulations reflect common preconceptions about generating processes regarding noise distributions, model classes, and more. In this work, we propose a novel method for falsifying the output of a causal discovery algorithm in the absence of ground truth. Our key insight is that while statistical learning seeks stability across subsets of data points, causal learning should seek stability across subsets of variables. Motivated by this insight, our method relies on a notion of compatibility between causal graphs learned on different subsets of variables. We prove that detecting incompatibilities can falsify wrongly inferred causal relations due to violation of assumptions or errors from finite sample effects. Although passing such compatibility tests is only a necessary criterion for good performance, we argue that it provides strong evidence for the causal models whenever compatibility entails strong implications for the joint distribution. We also demonstrate experimentally that detection of incompatibilities can aid in causal model selection.
Publishing Year
Date Published
2023-07-18
Journal Title
arXiv
Article Number
2307.09552
IST-REx-ID

Cite this

Faller PM, Vankadara LC, Mastakouri AA, Locatello F, Janzing D. Self-compatibility: Evaluating causal discovery without ground truth. arXiv. doi:10.48550/arXiv.2307.09552
Faller, P. M., Vankadara, L. C., Mastakouri, A. A., Locatello, F., & Janzing, D. (n.d.). Self-compatibility: Evaluating causal discovery without ground truth. arXiv. https://doi.org/10.48550/arXiv.2307.09552
Faller, Philipp M., Leena Chennuru Vankadara, Atalanti A. Mastakouri, Francesco Locatello, and Dominik Janzing. “Self-Compatibility: Evaluating Causal Discovery without Ground Truth.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2307.09552.
P. M. Faller, L. C. Vankadara, A. A. Mastakouri, F. Locatello, and D. Janzing, “Self-compatibility: Evaluating causal discovery without ground truth,” arXiv. .
Faller PM, Vankadara LC, Mastakouri AA, Locatello F, Janzing D. Self-compatibility: Evaluating causal discovery without ground truth. arXiv, 2307.09552.
Faller, Philipp M., et al. “Self-Compatibility: Evaluating Causal Discovery without Ground Truth.” ArXiv, 2307.09552, doi:10.48550/arXiv.2307.09552.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]

Link(s) to Main File(s)
Access Level
OA Open Access

Export

Marked Publications

Open Data ISTA Research Explorer

Sources

arXiv 2307.09552

Search this title in

Google Scholar