A0606
Title: Empirical Bayes estimation: When does g-modelling beat f-modelling in theory (and in practice)?
Authors: Yandi Shen - University of Chicago (United States) [presenting]
Yihong Wu - Yale University (United States)
Abstract: Empirical Bayes (EB) is a popular framework for the large-scale inference that aims to find data-driven estimators to compete with the Bayesian oracle that knows the truth prior. Two principled approaches to EB estimation have emerged over the years: f-modelling, which constructs an approximate Bayes rule by estimating the marginal distribution of the data, and g-modelling, which estimates the prior data and then applies the learned Bayes rule. For the Poisson model, the prototypical examples are the celebrated Robbins estimator and the nonparametric MLE (NPMLE), respectively. It has long been recognized in practice that while being conceptually appealing and computationally simple, Robbin's estimator lacks robustness and can be easily derailed by "outliers" (data points rarely observed before). A theoretical justification for the superiority of NPMLE over Robbins for heavy-tailed data is provided by considering priors with bounded pth moment previously studied for the Gaussian model. For the Poisson model with sample size n, assuming p>1 (for otherwise triviality arises), it is shown that the NPMLE with appropriate regularization and truncation achieves a total regret $O(n^{3/(2p+1)})$, which is minimax optimal within logarithmic factors. In contrast, the total regret of Robbin's estimator (with similar truncation) is $O(n^{3/(p+2)})$ and hence suboptimal by a polynomial factor.
