Detecting causal associations in time series datasets is a key challenge for novel insights into complex dynamical systems such as the Earth system or the human brain. Interactions in such systems present a number of major challenges for causal discovery techniques and it is largely unknown which methods perform best for which challenge.
The CauseMe platform provides ground truth benchmark datasets featuring different real data challenges to assess and compare the performance of causal discovery methods. The available benchmark datasets are either generated from synthetic models mimicking real challenges, or are real world data sets where the causal structure is known with high confidence. The datasets vary in dimensionality, complexity and sophistication.
There are two ways to contribute:
Don't just label me. Cause Me!
Causeme currently covers a wide range of synthetic model data mimicking a number of real world challenges. These cover time delays, autocorrelation, nonlinearity, chaotic dynamics, extreme events, measurement error, and will be extended by many more. Method developers can upload their predictions (matrices of causal connections) and the platform evaluates and ranks the methods according to different metrics of performance. After registering and logging in, more information, datasets, and example code snippets are given.
If you find the platform useful, acknowledge it by citing these references:
Inferring causation from time series in Earth system sciences. J. Runge, S. Bathiany, E. Bollt, G. Camps-Valls, D. Coumou, E. Deyle, C. Glymour, M. Kretschmer, M.D. Mahecha, J. Munoz-Mari, E.H. van Ness, J. Peters, R. Quax, M. Reichstein, M. Scheffer, B. Schölkopf, P. Spirtes, G. Sugihara, J. Sun, K. Zhang, J. Zscheischler. Nature Communications 10: 2553 (2019).
Detecting causal associations in large nonlinear time series datasets. J. Runge, P. Nowack, M. Kretschmer, S. Flaxman, and D. Sejdinovic. Science Advances 5(11): eaau4996 (2019).
Causal network reconstruction from time series: From theoretical assumptions to practical estimation. J. Runge. Chaos: An Interdisciplinary Journal of Nonlinear Science, 28:7 (2018).
CauseMe: An online system for benchmarking causal discovery methods. J. Muñoz-Marí, G. Mateo, J. Runge, and G. Camps-Valls. In preparation (2020).
Further causality benchmark projects: