A0452
Title: Statistical analysis on in-context learning
Authors: Masaaki Imaizumi - The University of Tokyo (Japan) [presenting]
Abstract: Deep learning and artificial intelligence technologies have made great progress, and the usage of foundation models has attracted strong attention to their general ability. Motivated by this fact, mathematical understanding is required to efficiently control and develop these technologies. A statistics-based analysis of a scheme called in-context learning is presented, which is a useful framework of meta-learning to describe foundation models. It is argued that in-context learning can efficiently learn the latent structure of the data, using the property of transformers used in the learning scheme can efficiently handle the distribution of observations.
