B1753
Title: A stochastic sampling method to enable efficient training
Authors: Hellen Xie - Apple (United States) [presenting]
Abstract: A sampling method is introduced to enable efficient training. In the era of large models, most of them require training procedures using giant amounts of training data, usually at a billion level or even more. However, there can be various overlapping or contaminations in data that cause models to waste time on learning. Hence, a stochastic sampling method is shown that can help avoid such waste, in which step-wise, random sample data is selected using adjusted weights of training data from calculating its contamination percentage and increased losses.
