$\begingroup$

I was reading the book Deep Learning by Goodfellow, Bengio & Courville and it seemed to imply that samples are only useful to:

approximate sum/integrals (why is this sooo important?) when the goal is to generate samples itself (which is trivial since of course its important to sample if that is the goal)

However, I remain unable to appreciate why sample is so important and why so much hard work has gone to study such a topic. Why is sampling important? Are there no other motivations? Are these really important enough on its own?

I'd love to be able to appreciate why sampling is an important topic.

My own thoughts

As someone inclined for ML, minimizing the expected loss is my goal:

$$ E_{x,y \sim p*_{x,y}} [Loss(f(X),Y)]$$

where $p^*$ is the true unknown distribution.

so I guess since this expectation is a sum or an integral we could try approximating the true generalization of our model if we could create more samples or create a model of the true distribution. This seems important, though it seems that this is not the approach people do for ML for some reason...

To provide further context on the exact extract I was reading from the deep learning book on the chapter on sampling (and Monte Carlo Methods) here is exact paragraph I was reading, title:

Why Sampling:

There are many reasons that we may wish to draw samples from a probability distribution. Sampling provides a ﬂexible way to approximate many sums and integrals at reduced cost. Sometimes we use this to provide a signiﬁcant speedup toa costly but tractable sum, as in the case when we subsample the full training costwith minibatches. In other cases, our learning algorithm requires us to approximatean intractable sum or integral, such as the gradient of the log partition function ofan undirected model. In many other cases, sampling is actually our goal, in thesense that we want to train a model that can sample from the training distribution. (Chapter 17)

for me just reading that section equates to "drawing samples (from a model) is only useful to approximate sums/integrals and when you want to do sampling". For someone with much less of a statistics background, this justification seems quite shallow. I have seen a lot of mathematics and textbooks (like Koller's PGM book) devoted to sampling from models. This seems quite an important topic and it just seems that this book lacked a proper motivation for the why. This is where my question stems from.