Chapter 6 Four Examples

This chapter treats four examples of non-trivial statistical models in some detail. These are all parametric models, and a central computational challenge is to fit the models to data via (penalized) likelihood maximization. The actual optimization algorithms and implementations are the topics of Chapters 7 and 8. The focus of this chapter is on the structure of the statistical models themselves to provide the necessary background for the later chapters.

Statistical models come in all forms and shapes, and it is possible to take a very general and abstract mathematical approach; statistical models are parametrized families of probability distributions. To say anything of interest, we need more structure such as structure on the parameter set, properties of the parametrized distributions, and properties of the mapping from the parameter set to the distributions. For any specific model we have ample of structure but often also an overwhelming amount of irrelevant details that will be more distracting than clarifying. The intention is that the four examples treated will illustrate the breath of statistical models that share important structures without getting lost in a wasteland of abstractions.

If one should emphasize a single abstract idea that is of theoretical value as well as of practical importance, it is the idea of exponential families. Statistical models that are exponential families have so much structure that the general theory provides a number of results and details of practical value for individual models. Exponential families are exemplary statistical models, that are widely used as models of data, or as central building blocks of more complicated models of data. For this reason, the treatment of the examples is preceded by a treatment of exponential families.