Theory of Statistics.

MJ Schervish.

Springer, 1995.

[MathSciNet]

A very good reference on abstract Bayesian methods, exchangeability, sufficiency, and parametric models (including infinite-dimensional Bayesian models) are the first two chapters of Schervish's Theory of Statistics.

Posterior convergence A clear and readable introduction to the questions studied in this area, and to how they are addressed, is a survey chapter by Ghosal which is Bayesian Nonparametrics.

JK Ghosh and RV Ramamoorthi.

Springer, 2002.

[MathSciNet]

The following sample references are a small subset of the large and growing literature on this subject: Misspecification in infinite-dimensional Bayesian statistics.

BJK Kleijn and AW van der Vaart.

Annals of Statistics, 34(2):837-877, 2006.

[MathSciNet]



[MathSciNet] Posterior convergence rates of Dirichlet mixtures at smooth densities.

S Ghosal and AW van der Vaart.

Annals of Statistics, 35(2):697-723, 2007.

[MathSciNet]



[MathSciNet] Rates of contraction of posterior distributions based on Gaussian process priors.

AW van der Vaart and JH van Zanten.

Annals of Statistics, 36(3):1435-1463, 2008.

[MathSciNet]



[MathSciNet] A semiparametric Bernstein-von Mises theorem for Gaussian process priors.

I Castillo.

Probability Theory and Related Fields, 152:53-99, 2012.

[PDF]

A clear and readable introduction to the questions studied in this area, and to how they are addressed, is a survey chapter by Ghosal which is referenced above . The following monograph is a good reference that provides many more details. Be aware though that the most interesting work in this area has arguably been done in the past decade, and hence is not covered by the book.The following sample references are a small subset of the large and growing literature on this subject:

Exchangeability For a good introduction to exchangeability and its implications for Bayesian models, see Schervish's Theory of Statistics, which is Exchangeability and continuum limits of discrete random structures.

DJ Aldous.

In Proceedings of the International Congress of Mathematicians, 2010.

[PDF]

The most comprehensive and rigorous treatise on exchangeability I am aware of is: Probabilistic Symmetries and Invariance Principles.

O Kallenberg.

Springer, 2005.

[MathSciNet]

I discuss applications to nonparametric Bayesian models of data not representable as exchangeable sequences in this preprint: Nonparametric priors on complete separable metric spaces.

P Orbanz.

Preprint.

[PDF]

For a good introduction to exchangeability and its implications for Bayesian models, see Schervish's Theory of Statistics, which is referenced above . If you are interested in the bigger picture, and in how exchangeability generalizes to other random structures than exchangeable sequences, I highly recommend an article based on David Aldous' lecture at the International Congress of Mathematicians:The most comprehensive and rigorous treatise on exchangeability I am aware of is:I discuss applications to nonparametric Bayesian models of data not representable as exchangeable sequences in this preprint:

Urns and power laws When the Dirichlet process was first developed,



For Bayesian nonparametrics, urns provide a probabilistic tool to study the sizes of clusters in a clustering model, or more generally the weight distributions of random discrete measures. They also provide a link to population genetics, where urns model the distribution of species; you will sometimes encounter references to species sampling models. The relationship between the different terminologie is \[\begin{aligned} \text{colors in urn } = \text{ species } = \text{ clusters } \end{aligned} \] and \[\begin{aligned} \#\text{balls } = \#\text{individuals } = \text{ cluster size. } \end{aligned} \] A key property of Pólya urns is that they can generate power law distributions, which occur in applications such as language models or social networks.



If you are interested in urns and power laws, I recommend that you have a look at the following two survey articles (in this order): Notes on the occupancy problem with infinitely many boxes: General asymptotics and power laws.

AV Gnedin, B Hansen, and J Pitman.

Probability Surveys, 4:146-171, 2007.

[PDF]



[PDF] A survey of random processes with reinforcement.

R Pemantle.

Probability Surveys, 4:1-79, 2007.

[PDF]

When the Dirichlet process was first developed, Blackwell and MacQueen realized that a sample from a DP can be generated by a so-called Pólya urn with infinitely many colors. Roughly speaking, an urn model assumes that balls of different colors are contained in an urn, and are drawn uniformly at random; the proportions of balls per color determine the probability of each color to be drawn. A specific urn is defined by a rule for how the number of balls is changed when a color is drawn. In Pólya urns, the number of balls of a color is increased whenever that color is drawn; this process is called reinforcement, and corresponds to the rich-get-richer property of the Dirichlet process. There are many different versions of Pólya urns, defined by different reinforcement rules.For Bayesian nonparametrics, urns provide a probabilistic tool to study the sizes of clusters in a clustering model, or more generally the weight distributions of random discrete measures. They also provide a link to population genetics, where urns model the distribution of species; you will sometimes encounter references to species sampling models. The relationship between the different terminologie is \[\begin{aligned} \text{colors in urn } = \text{ species } = \text{ clusters } \end{aligned} \] and \[\begin{aligned} \#\text{balls } = \#\text{individuals } = \text{ cluster size. } \end{aligned} \] A key property of Pólya urns is that they can generate power law distributions, which occur in applications such as language models or social networks.If you are interested in urns and power laws, I recommend that you have a look at the following two survey articles (in this order):