hand!

Suppose you’ve tossed a coin 1,000 times and obtained 292. You’d like to know what the Probability of obtaining heads is from a single coin toss – but you don’t just want a single estimate, you want an entire distribution. If you define
- y: the number of heads you obtain
- θ: the probability of obtaining heads from a single coin toss
and then model y as a binomial distribution with n=1,000, then the posterior distribution is very easy to obtain with just a few lines of code:

But…could we have calculated this distribution analytically without PyMC3’s help?
Bayes’ theorem
From Bayes’ theorem, we have:

Applying this to our problem, we obtain

Substituting the definitions of the two distributions, we can re-write the right-hand-side as

Removing constants (that don’t depend on theta), we can substitute this into the equation above to obtain

which we can recognise (by inspection) as a Beta(292+1, 1000–292+1) distribution. Because our posterior and our prior are from the same family, the Beta distribution is known as a _conjugate prior_ for the Binomial distribution.
If we try plotting this using scipy, we’ll get the same distribution that PyMC3 has calculated for us:

That’s the nice thing about conjugate priors: they make posterior probabilities analytically tractable.
Calculating posteriors by hand is an excellent exercise in deepening your understanding. However, in general, we will not find ourselves in such fortunate situations – that’s when we need tools such as PyMC3.
Further reading
For an excellent and practical guide to Bayesian statistics, I recommend reading Bayesian Analysis with Python and listening to the Learning Bayesian Statistics podcast.