A1591
Title: Consensus Monte Carlo for mixtures of categorical distributions
Authors: Julie Fendler - University of Cambridge (United Kingdom) [presenting]
Paul Kirk - University of Cambridge (United Kingdom)
Abstract: Motivated by privacy-sensitive electronic health record (EHR) data, a novel Consensus Monte Carlo (CMC) algorithm for Bayesian mixture models in a federated learning setting is presented, addressing scenarios where data cannot be fully shared or pooled across compute nodes. CMC assumes that the dataset is split into $S$ shards, and an MCMC algorithm is run independently within each shard. For a given shard, the target posterior of the MCMC algorithm, called the local posterior, is proportional to the likelihood computed within the shard multiplied by a fractionated prior. To draw samples from the global posterior (i.e., the posterior over the entire dataset), different methods exist for combining the local posteriors. A naive approach is to average the draws from the local posteriors at each iteration. Alternatively, since the data may be heterogeneously distributed across shards, a weighted mean can be considered. Prior work proposes to infer these weights, called aggregation weights, within a variational inference framework. However, in the context of finite mixture models, existing approaches assume that the number of clusters and the mixture weights are known. Such methods are extended to more general settings and considers an application to large-scale EHR data.
