CMStatistics 2016: Start Registration
View Submission - CMStatistics
B0933
Title: Causal transfer in machine learning Authors:  Mateo Rojas Carulla - Cambridge / Max Planck for Intelligent Systems (Germany) [presenting]
Bernhard Scholkopf - Max Planck Institute for Intelligent Systems (Germany)
Richard Turner - University of Cambridge (United Kingdom)
- ()
Abstract: Traditional methods for machine learning assume that the training and test data are drawn independently from the same probability distribution. Transfer learning aims to go beyond this paradigm and considers settings in which the data at training and testing can come from different probability distributions, referred to as tasks. Methods of transfer learning try to combine knowledge from several related tasks (or domains) to improve performance on a test task. Inspired by causal methodology, we relax the usual covariate shift assumption and assume that it holds true for a subset of predictor variables: the conditional distribution of the target variable given this subset of predictors is invariant over all tasks. We prove that in an adversarial setting using this subset for prediction is optimal if no examples from the test task are observed. If examples from the test task are available, we provide a method to transfer knowledge from the training tasks and exploit all available predictors for prediction. We introduce a practical method which allows for automatic inference of the above subset. We present results on synthetic data sets and a gene deletion data set.