[edit]

WASP: Scalable Bayes via barycenters of subset posteriors

Sanvesh Srivastava, Volkan Cevher, Quoc Dinh, David Dunson

; Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, PMLR 38:912-920, 2015.

Abstract

The promise of Bayesian methods for big data sets has not fully been realized due to the lack of scalable computational algorithms. For massive data, it is necessary to store and process subsets on different machines in a distributed manner. We propose a simple, general, and highly efficient approach, which first runs a posterior sampling algorithm in parallel on different machines for subsets of a large data set. To combine these subset posteriors, we calculate the Wasserstein barycenter via a highly efficient linear program. The resulting estimate for the Wasserstein posterior (WASP) has an atomic form, facilitating straightforward estimation of posterior summaries of functionals of interest. The WASP approach allows posterior sampling algorithms for smaller data sets to be trivially scaled to huge data. We provide theoretical justification in terms of posterior consistency and algorithm efficiency. Examples are provided in complex settings including Gaussian process regression and nonparametric Bayes mixture models.

Cite this Paper

BibTeX @InProceedings{pmlr-v38-srivastava15, title = {{WASP: Scalable Bayes via barycenters of subset posteriors}}, author = {Sanvesh Srivastava and Volkan Cevher and Quoc Dinh and David Dunson}, pages = {912--920}, year = {2015}, editor = {Guy Lebanon and S. V. N. Vishwanathan}, volume = {38}, series = {Proceedings of Machine Learning Research}, address = {San Diego, California, USA}, month = {09--12 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v38/srivastava15.pdf}, url = {http://proceedings.mlr.press/v38/srivastava15.html}, abstract = {The promise of Bayesian methods for big data sets has not fully been realized due to the lack of scalable computational algorithms. For massive data, it is necessary to store and process subsets on different machines in a distributed manner. We propose a simple, general, and highly efficient approach, which first runs a posterior sampling algorithm in parallel on different machines for subsets of a large data set. To combine these subset posteriors, we calculate the Wasserstein barycenter via a highly efficient linear program. The resulting estimate for the Wasserstein posterior (WASP) has an atomic form, facilitating straightforward estimation of posterior summaries of functionals of interest. The WASP approach allows posterior sampling algorithms for smaller data sets to be trivially scaled to huge data. We provide theoretical justification in terms of posterior consistency and algorithm efficiency. Examples are provided in complex settings including Gaussian process regression and nonparametric Bayes mixture models.} } Copy to Clipboard Download

Endnote %0 Conference Paper %T WASP: Scalable Bayes via barycenters of subset posteriors %A Sanvesh Srivastava %A Volkan Cevher %A Quoc Dinh %A David Dunson %B Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2015 %E Guy Lebanon %E S. V. N. Vishwanathan %F pmlr-v38-srivastava15 %I PMLR %J Proceedings of Machine Learning Research %P 912--920 %U http://proceedings.mlr.press %V 38 %W PMLR %X The promise of Bayesian methods for big data sets has not fully been realized due to the lack of scalable computational algorithms. For massive data, it is necessary to store and process subsets on different machines in a distributed manner. We propose a simple, general, and highly efficient approach, which first runs a posterior sampling algorithm in parallel on different machines for subsets of a large data set. To combine these subset posteriors, we calculate the Wasserstein barycenter via a highly efficient linear program. The resulting estimate for the Wasserstein posterior (WASP) has an atomic form, facilitating straightforward estimation of posterior summaries of functionals of interest. The WASP approach allows posterior sampling algorithms for smaller data sets to be trivially scaled to huge data. We provide theoretical justification in terms of posterior consistency and algorithm efficiency. Examples are provided in complex settings including Gaussian process regression and nonparametric Bayes mixture models. Copy to Clipboard Download

RIS TY - CPAPER TI - WASP: Scalable Bayes via barycenters of subset posteriors AU - Sanvesh Srivastava AU - Volkan Cevher AU - Quoc Dinh AU - David Dunson BT - Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics PY - 2015/02/21 DA - 2015/02/21 ED - Guy Lebanon ED - S. V. N. Vishwanathan ID - pmlr-v38-srivastava15 PB - PMLR SP - 912 DP - PMLR EP - 920 L1 - http://proceedings.mlr.press/v38/srivastava15.pdf UR - http://proceedings.mlr.press/v38/srivastava15.html AB - The promise of Bayesian methods for big data sets has not fully been realized due to the lack of scalable computational algorithms. For massive data, it is necessary to store and process subsets on different machines in a distributed manner. We propose a simple, general, and highly efficient approach, which first runs a posterior sampling algorithm in parallel on different machines for subsets of a large data set. To combine these subset posteriors, we calculate the Wasserstein barycenter via a highly efficient linear program. The resulting estimate for the Wasserstein posterior (WASP) has an atomic form, facilitating straightforward estimation of posterior summaries of functionals of interest. The WASP approach allows posterior sampling algorithms for smaller data sets to be trivially scaled to huge data. We provide theoretical justification in terms of posterior consistency and algorithm efficiency. Examples are provided in complex settings including Gaussian process regression and nonparametric Bayes mixture models. ER - Copy to Clipboard Download

APA Srivastava, S., Cevher, V., Dinh, Q. & Dunson, D.. (2015). WASP: Scalable Bayes via barycenters of subset posteriors. Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, in PMLR 38:912-920 Copy to Clipboard Download

Related Material