Introduction to Statistical Computing

Jochen Voss, School of Mathematics, University of Leeds, UK.

• List of algorithms ixPreface xiNomenclature xiii1 Random number generation 11.1 Pseudo random number generators 21.1.1 The linear congruential generator 21.1.2 Quality of pseudo random number generators 41.1.3 Pseudo random number generators in practice 81.2 Discrete distributions 81.3 The inverse transform method 111.4 Rejection sampling 151.4.1 Basic rejection sampling 151.4.2 Envelope rejection sampling 181.4.3 Conditional distributions 221.4.4 Geometric interpretation 261.5 Transformation of random variables 301.6 Special-purpose methods 361.7 Summary and further reading 36Exercises 372 Simulating statistical models 412.1 Multivariate normal distributions 412.2 Hierarchical models 452.3 Markov chains 502.3.1 Discrete state space 512.3.2 Continuous state space 562.4 Poisson processes 582.5 Summary and further reading 67Exercises 673 Monte Carlo methods 693.1 Studying models via simulation 693.2 Monte Carlo estimates 743.2.1 Computing Monte Carlo estimates 753.2.2 Monte Carlo error 763.2.3 Choice of sample size 803.2.4 Refined error bounds 823.3 Variance reduction methods 843.3.1 Importance sampling 843.3.2 Antithetic variables 883.3.3 Control variates 933.4 Applications to statistical inference 963.4.1 Point estimators 973.4.2 Confidence intervals 1003.4.3 Hypothesis tests 1033.5 Summary and further reading 106Exercises 1064 Markov Chain Monte Carlo methods 1094.1 The Metropolis–Hastings method 1104.1.1 Continuous state space 1104.1.2 Discrete state space 1134.1.3 Random walk Metropolis sampling 1164.1.4 The independence sampler 1194.1.5 Metropolis–Hastings with different move types 1204.2 Convergence of Markov Chain Monte Carlo methods 1254.2.1 Theoretical results 1254.2.2 Practical considerations 1294.3 Applications to Bayesian inference 1374.4 The Gibbs sampler 1414.4.1 Description of the method 1414.4.2 Application to parameter estimation 1464.4.3 Applications to image processing 1514.5 Reversible Jump Markov Chain Monte Carlo 1584.5.1 Description of the method 1604.5.2 Bayesian inference for mixture distributions 1714.6 Summary and further reading 1784.6 Exercises 1785 Beyond Monte Carlo 1815.1 Approximate Bayesian Computation 1815.1.1 Basic Approximate Bayesian Computation 1825.1.2 Approximate Bayesian Computation with regression 1885.2 Resampling methods 1925.2.1 Bootstrap estimates 1925.2.2 Applications to statistical inference 1975.3 Summary and further reading 209Exercises 2096 Continuous-time models 2136.1 Time discretisation 2136.2 Brownian motion 2146.2.1 Properties 2166.2.2 Direct simulation 2176.2.3 Interpolation and Brownian bridges 2186.3 Geometric Brownian motion 2216.4 Stochastic differential equations 2246.4.1 Introduction 2246.4.2 Stochastic analysis 2266.4.3 Discretisation schemes 2316.4.4 Discretisation error 2366.5 Monte Carlo estimates 2436.5.1 Basic Monte Carlo 2436.5.2 Variance reduction methods 2476.5.3 Multilevel Monte Carlo estimates 2506.6 Application to option pricing 2556.7 Summary and further reading 259Exercises 260Appendix A Probability reminders 263A.1 Events and probability 263A.2 Conditional probability 266A.3 Expectation 268A.4 Limit theorems 269A.5 Further reading 270Appendix B Programming in R 271B.1 General advice 271B.2 R as a Calculator 272B.2.1 Mathematical operations 273B.2.2 Variables 273B.2.3 Data types 275B.3 Programming principles 282B.3.1 Don’t repeat yourself! 283B.3.2 Divide and conquer! 286B.3.3 Test your code! 290B.4 Random number generation 292B.5 Summary and further reading 294Exercises 294Appendix C Answers to the exercises 299C.1 Answers for Chapter 1 299C.2 Answers for Chapter 2 315C.3 Answers for Chapter 3 319C.4 Answers for Chapter 4 328C.5 Answers for Chapter 5 342C.6 Answers for Chapter 6 350C.7 Answers for Appendix B 366References 375Index 379

"The exposition is quite clear, intuitive, and is a useful complement to more abstract treatises on stochastic calculus and simulation."   (MathSciNet, 1 December 2015)“Careful presentation and examples make this book accessible to a wide range of students and suitable for self-study or as the basis of a taught course.”  (Zentralblatt MATH, 1 March 2014)“Statistical computing in its broadest sense is an ever-growing field far too extensive to be covered in a single text. The current book has a far more manageable scope, notwithstanding its title. Its focus is on the use of Monte Carlo methods to simulate random systems and explore statistical models.”  (Mathematical Association of America, 1 January 2014)