Book review Data analysis a Bayesian tutorial
The second edition of this book is written by Devinder Sivia (chapters 1 through 8) with John Skilling (chapters 9 and 10). The book gives a concise (in 250 pages), but full description of Bayesian data analysis. I know both authors personally.
Data Analysis. A Bayesian tutorial
The first chapter gives a clear, concise history of Bayesian statistics and describes the distinction with classical statistics.
The second chapter starts with the question “is this a fair coin?”, after you have seen 4 heads with a coin in 11 tosses. With this question at hand, you will be guided along the binomial distribution, prior distributions, sequential analysis, parameter estimation, error limits and confidence intervals, the Gauss distribution and even the Cauchy distribution. All in an entertaining style.
Chapter three builds on this with multivariate analysis, correlations, multimodal distributions and non-linear solution methods. Furthermore, this chapter deals with coordinate transformations of distributions.
In chapter four the search for a good model is undertaken. How the pitfalls of over-fitting can be avoided by the use of priors. Model selection is the most important application of Bayesian Statistics. Here Rev. Thomas Bayes shows his true power.
What is a prior? How do you choose this? What is the role of entropy in here? These questions are discussed in the fifth chapter. In short, the prior describes the relevant professional background about the problem.
The chapters six, seven and eight describe various advanced subjects, such as image reconstruction, spatial correlations, setting up optimal experiments, instrumental calibration, noise in data, the treatment of outliers, zero point correction, and so on.
I am less enthusiastic about Skilling’s chapters 9 and 10. Skilling is the inventor of the Nested Sampling algorithm, one of the most important developments in computer science. In chapter nine the principles are explained, and in chapter ten details about the MCMC algorithm are given.
His example of the lighthouse is beautiful. But this is followed by an almost incomprehensible example about the number of “gates” needed to pass an “object”. Other examples are also didactic insufficiently thought out.
Pros: a concise book in an entertaining, clear style with relatively simple mathematical derivations. Good to get an overview of Bayesian data analysis in a short time. Skilling makes a number of C programs publicly available to experiment with.
Cons: Unusual notation, such as prob (A | B), for probabilities. The theory at Skilling is quickly confusing because the same symbol L is used for two different aspects and the important contour (around the prior mass) has no mathematical symbol at all. There is little connection between Sivos and Skilling’s parts of the book.
However, Sivia’s chapters make this book so good that it gets the highest score ★★★★★.