A Student’s Guide to Bayesian Statistics

Ben Lambert is a researcher at Imperial College London where he works on the epidemiology of malaria. He has worked in applied statistical inference for about a decade, formerly at the University of Oxford, and is the author of over 500 online lectures on econometrics and statistics. He also somewhat strangely went to school in Thomas Bayes’ home town for many years, Tunbridge Wells.

• Chapter 1: How to best use this bookThe purpose of this bookWho is this book for?Pre-requisitesBook outlineRoute planner - suggested journeys through BayeslandVideoProblem setsCodeR and StanWhy don’t more people use Bayesian statistics?What are the tangible (non-academic) benefits of Bayesian statistics?Part I: An introduction to Bayesian inferenceChapter 2: The subjective worlds of Frequentist and Bayesian statisticsBayes’ rule - allowing us to go from the effect back to its causeThe purpose of statistical inferenceThe world according to FrequentistsThe world according to BayesiansDo parameters actually exist and have a point value?Frequentist and Bayesian inferenceBayesian inference via Bayes’ ruleImplicit versus Explicit subjectivityChapter 3: Probability - the nuts and bolts of Bayesian inferenceProbability distributions: helping us explicitly state our ignoranceIndependenceCentral Limit TheoremsA derivation of Bayes’ ruleThe Bayesian inference process from the Bayesian formulaPart II: Understanding the Bayesian formulaChapter 4: LikelihoodsWhat is a likelihood?Why use ‘likelihood’ rather than ‘probability’?What are models and why do we need them?How to choose an appropriate likelihood?Exchangeability vs random samplingMaximum likelihood - a short introductionChapter 5: PriorsWhat are priors, and what do they represent?The explicit subjectivity of priorsCombining a prior and likelihood to form a posteriorConstructing priorsA strong model is less sensitive to prior choiceChapter 6: The devil’s in the denominatorAn introduction to the denominatorThe difficulty with the denominatorHow to dispense with the difficulty: Bayesian computationChapter 7: The posterior - the goal of Bayesian inferenceExpressing parameter uncertainty in posteriorsBayesian statistics: updating our pre-data uncertaintyThe intuition behind Bayes’ rule for inferencePoint parameter estimatesIntervals of uncertaintyFrom posterior to predictions by samplingPart III: Analytic Bayesian methodsChapter 8: An introduction to distributions for the mathematically-un-inclinedThe interrelation among distributionsSampling distributions for likelihoodsPrior distributionsHow to choose a likelihoodTable of common likelihoods, their uses, and reasonable priorsDistributions of distributions, and mixtures - link to website, and relevanceChapter 9: Conjugate priors and their place in Bayesian analysisWhat is a conjugate prior and why are they useful?Gamma-poisson exampleNormal example: giraffe heightTable of conjugate priorsThe lessons and limits of a conjugate analysisChapter 10: Evaluation of model fit and hypothesis testingPosterior predictive checksWhy do we call it a p value?Statistics measuring predictive accuracy: AIC, Deviance, WAIC and LOO-CVMarginal likelihoods and Bayes factorsChoosing one model, or a number?Sensitivity analysisChapter 11: Making Bayesian analysis objective?The illusion of the ’uninformative’ uniform priorJeffreys’ priorsReference priorsEmpirical BayesA move towards weakly informative priorsPart IV: A practical guide to doing real life Bayesian analysis: Computational BayesChapter 12: Leaving conjugates behind: Markov Chain Monte CarloThe difficulty with real life Bayesian inferenceDiscrete approximation to continuous posteriorsThe posterior through quadratureIntegrating using independent samples: an introduction to Monte CarloWhy is independent sampling easier said than done?Ideal sampling from a posterior using only the un-normalised posteriorMoving from independent to dependent samplingWhat’s the catch with dependent samplers?Chapter 13: Random Walk MetropolisSustainable fishingProspecting for goldDefining the Metropolis algorithmWhen does Metropolis work?Efficiency of convergence: the importance of choosing the right proposal scaleMetropolis-HastingsJudging convergenceEffective sample size revisitedChapter 14: Gibbs samplingBack to prospecting for goldDefining the Gibbs algorithmGibbs’ earth: the intuition behind the Gibbs algorithmThe benefits and problems with Gibbs and Random Walk MetropolisA change of parameters to speed up explorationChapter 15: Hamiltonian Monte CarloHamiltonian Monte Carlo as a sledgeNLP spaceSolving for the sledge motion over NLP spaceHow to shove the sledgeThe acceptance probability of HMCThe complete Hamiltonian Monte Carlo algorithmThe performance of HMC versus Random Walk Metropolis and GibbsOptimal step length of HMC: introducing the “No U-Turn Sampler”Chapter 16: StanWhy Stan, and how to get itGetting setup with Stan using RStanOur first words in StanEssential Stan readingWhat to do when things go wrongHow to get further helpPart V: Hierarchical models and regressionChapter 17: Hierarchical modelsThe spectrum from fully-pooled to heterogeneousNon-centered parameterisations in hierarchical modelsCase study: Forecasting the EU referendum resultThe importance of fake data simulation for complex modelsChapter 18: Linear regression modelsExample: high school test scores in EnglandPooled modelInteractionsHeterogeneous coefficient modelHierarchical modelIncorporating LEA-level dataChapter 19: Generalised linear models and other animalsExample: electoral participation in European countriesDiscrete parameter models in Stan

An excellent resource on Bayesian analysis accessible to students from a diverse range of statistical backgrounds and interests.  Easy to follow with well documented examples to illustrate key concepts.