Learning Representation with Causal Invariance (ICLR 2019)

This talk was first given as a ICLR 2019 keynote.

Joint work with Martin Arjovsky (NYU), Ishaan Gulrajani (Google), and David Lopez-Paz (Facebook)

Learning algorithms often capture spurious correlations present in the training data distribution instead of addressing the task of interest. Such spurious correlations occur because the data collection process is subject to uncontrolled confounding biases. Suppose however that we have access to multiple datasets exemplifying the same concept but whose distributions exhibit different biases. Can we learn something that is common across all these distributions, while ignoring the spurious ways in which they differ? This can be achieved by projecting the data into a representation space that satisfy a causal invariance criterion. This idea differs in important ways from previous work on statistical robustness or adversarial objectives. Similar to recent work on invariant feature selection, this is about discovering the actual mechanism underlying the data instead of modeling its superficial statistics.

• Video recording (without the slides) (from 12:00 to 50:00)
• Slides (djvu 3.4M) (pdf, 8M)