A0757
Title: Random-walk debiased inference for contextual ranking model with application in large language model evaluation
Authors: Yichi Zhang - Statistics Department, Indiana University Bloomington (United States) [presenting]
Abstract: A debiased inference framework is proposed to infer the ranking structure in the contextual Bradley-Terry-Luce (BTL) model. A nonparametric maximum likelihood estimation method is first adopted using ReLU neural networks to estimate unknown preference functions in the model. For the inference of pairwise ranking, a novel random-walk debiased estimator is introduced, that efficiently aggregates all accessible estimating scores. In particular, under mild conditions, the debiased estimator yields a tractable distribution and achieves the semiparametric efficiency bound asymptotically. The method is further extended by incorporating multiplier bootstrap techniques for the uniform inference of ranking structures, and adapting it to accommodate the distributional shift of contextual variables. Thorough numerical studies are provided to validate the statistical properties of the method, and showcase its applicability in evaluating large language models based on human preferences under different contexts.
