CFE 2019: Start Registration
View Submission - CMStatistics
B0437
Title: Weighted empirical risk minimization: Transfer learning based on importance sampling Authors:  Charles Tillier - Telecom ParisTech (France) [presenting]
Stephan Clemencon - Telecom ParisTech (France)
Robin Vogel - Telecom ParisTech (France)
Mastane Achab - Telecom ParisTech (France)
Abstract: Statistical learning problems are considered when the distribution \$P'\$ of the training observations \$Z'_1,\; \ldots,\; Z'_n\$ differs from the distribution \$P\$ involved in the risk one seeks to minimize (referred to as the test distribution) but is still defined on the same measurable space as \$P\$ and dominates it. In the unrealistic case where the likelihood ratio \$\Phi(z)=dP/dP'(z)\$ is known, one may extend the Empirical Risk Minimization (ERM) approach to this specific transfer learning setup using the same idea as that behind Importance Sampling, by minimizing a weighted version of the empirical risk functional computed from the 'biased' training data \$Z'_i\$ with weights \$\Phi(Z'_i)\$. Although the importance function \$\Phi(z)\$ is generally unknown in practice, in various situations frequently encountered in practice, it takes a simple form and can be directly estimated from the \$Z'_i\$'s and some auxiliary information on the statistical population \$P\$. Besides, we will see that the generalization capacity of the approach aforementioned is preserved when plugging the resulting estimates of the \$\Phi(Z'_i)\$'s into the weighted empirical risk.