Computational Statistics

GEOF H. GIVENS, PhD, is Associate Professor in the Department of Statistics at Colorado State University. He serves as Associate Editor for Computational Statistics and Data Analysis. His research interests include statistical problems in wildlife conservation biology including ecology, population modeling and management, and automated computer face recognition.JENNIFER A. HOETING, PhD, is Professor in the Department of Statistics at Colorado State University. She is an award-winning teacher who co-leads large research efforts for the National Science Foundation. She has served as associate editor for the Journal of the American Statistical Association and Environmetrics. Her research interests include spatial statistics, Bayesian methods, and model selection.Givens and Hoeting have taught graduate courses on computational statistics for nearly twenty years, and short courses to leading statisticians and scientists around the world.

• PREFACE xvACKNOWLEDGMENTS xvii1 REVIEW 11.1 Mathematical Notation 11.2 Taylor’s Theorem and Mathematical Limit Theory 21.3 Statistical Notation and Probability Distributions 41.4 Likelihood Inference 91.5 Bayesian Inference 111.6 Statistical Limit Theory 131.7 Markov Chains 141.8 Computing 17PART I OPTIMIZATION2 OPTIMIZATION AND SOLVING NONLINEAR EQUATIONS 212.1 Univariate Problems 222.2 Multivariate Problems 34Problems 543 COMBINATORIAL OPTIMIZATION 593.1 Hard Problems and NP-Completeness 593.2 Local Search 653.3 Simulated Annealing 683.4 Genetic Algorithms 753.5 Tabu Algorithms 85Problems 924 EM OPTIMIZATION METHODS 974.1 Missing Data, Marginalization, and Notation 974.2 The EM Algorithm 984.3 EM Variants 111Problems 121PART II INTEGRATION AND SIMULATION5 NUMERICAL INTEGRATION 1295.1 Newton–Côtes Quadrature 1295.2 Romberg Integration 1395.3 Gaussian Quadrature 1425.4 Frequently Encountered Problems 146Problems 1486 SIMULATION AND MONTE CARLO INTEGRATION 1516.1 Introduction to the Monte Carlo Method 1516.2 Exact Simulation 1526.3 Approximate Simulation 1636.4 Variance Reduction Techniques 180Problems 1957 MARKOV CHAIN MONTE CARLO 2017.1 Metropolis–Hastings Algorithm 2027.2 Gibbs Sampling 2097.3 Implementation 218Problems 2308 ADVANCED TOPICS IN MCMC 2378.1 Adaptive MCMC 2378.2 Reversible Jump MCMC 2508.3 Auxiliary Variable Methods 2568.4 Other Metropolis–Hastings Algorithms 2608.5 Perfect Sampling 2648.6 Markov Chain Maximum Likelihood 2688.7 Example: MCMC for Markov Random Fields 269Problems 279PART III BOOTSTRAPPING9 BOOTSTRAPPING 2879.1 The Bootstrap Principle 2879.2 Basic Methods 2889.3 Bootstrap Inference 2929.4 Reducing Monte Carlo Error 3029.5 Bootstrapping Dependent Data 3039.6 Bootstrap Performance 3159.7 Other Uses of the Bootstrap 3169.8 Permutation Tests 317Problems 319PART IV DENSITY ESTIMATION AND SMOOTHING10 NONPARAMETRIC DENSITY ESTIMATION 32510.1 Measures of Performance 32610.2 Kernel Density Estimation 32710.3 Nonkernel Methods 34110.4 Multivariate Methods 345Problems 35911 BIVARIATE SMOOTHING 36311.1 Predictor–Response Data 36311.2 Linear Smoothers 36511.3 Comparison of Linear Smoothers 37711.4 Nonlinear Smoothers 37911.5 Confidence Bands 38411.6 General Bivariate Data 388Problems 38912 MULTIVARIATE SMOOTHING 39312.1 Predictor–Response Data 39312.2 General Multivariate Data 413Problems 416DATA ACKNOWLEDGMENTS 421REFERENCES 423INDEX 457