Applied Linear Regression

SANFORD WEISBERG, PhD, is Professor of Statistics and Director of the Statistical Consulting Service in the School of Statistics at the University of Minnesota. He is also a coauthor of Applied Regression Including Computing and Graphics and An Introduction to Regression Graphics, both published by Wiley.

• 1 Scatterplots 11.1 Scatterplots 21.2 Mean Functions 91.3 Variance Functions 121.4 Summary Graph 121.5 Tools for Looking at Scatterplots 131.6 Scatterplot Matrices 151.7 Problems 172 Simple Linear Regression 212.1 Ordinary Least Squares Estimation 222.2 Least Squares Criterion 242.3 Estimating the Variance 𝜎2 262.4 Properties of Least Squares Estimates 272.5 Estimated Variances 282.6 Confidence Intervals and 𝑡-Tests 292.7 The Coefficient of Determination, 𝑅2 332.8 The Residuals 352.9 Problems 373 Multiple Regression 493.1 Adding a Regressor to a Simple Linear Regression Model 493.2 The Multiple Linear Regression Model 533.3 Predictors and Regressors 533.4 Ordinary Least Squares 573.5 Predictions, Fitted Values and Linear Combinations 653.6 Problems 664 Interpretation of Main Effects 714.1 Understanding Parameter Estimates 714.2 Dropping Regressors 814.3 Experimentation Versus Observation 844.4 Sampling from a Normal Population 864.5 More on 𝑅2 884.6 Problems 905 Complex Regressors 955.1 Factors 955.2 Many Factors 1055.3 Polynomial Regression 1065.4 Splines 1095.5 Principal Components 1125.6 Missing Data 1155.7 Problems 1186 Testing and Analysis of Variance 1296.1 𝐹-tests 1306.2 The Analysis of Variance 1346.3 Comparisons of Means 1386.4 Power and Non-null Distributions 1386.5 Wald Tests 1406.6 Interpreting Tests 1426.7 Problems 1457 Variances 1517.1 Weighted Least Squares 1517.2 Misspecified Variances 1577.3 General Correlation Structures 1627.4 Mixed Models 1637.5 Variance Stabilizing Transformations 1657.6 The Delta Method 1667.7 The Bootstrap 1687.8 Problems 1738 Transformations 1798.1 Transformation Basics 1798.2 A General Approach to Transformations 1858.3 Transforming the Response 1908.4 Transformations of Nonpositive Variables 1928.5 Additive Models 1928.6 Problems 1939 Regression Diagnostics 1999.1 The Residuals 1999.2 Testing for Curvature 2069.3 Nonconstant Variance 2089.4 Outliers 2089.5 Influence of Cases 2129.6 Normality Assumption 2189.7 Problems 22010 Variable Selection 22710.1 Variable Selection and Parameter Assessment 22810.2 Variable Selection for Discovery 23010.3 Model Selection for Prediction 23810.4 Problems 24111 Nonlinear Regression 24511.1 Estimation for Nonlinear Mean Functions 24611.2 Inference Assuming Large Samples 24911.3 Starting Values 24911.4 Bootstrap Inference 25511.5 Further Reading 25711.6 Problems 25812 Binomial and Poisson Regression 26312.1 Distributions for Counted Data 26312.2 Regression Models For Counts 26512.3 Poisson Regression 27112.4 Transferring What You Know about Linear Models 27612.5 Generalized Linear Models 27812.6 Problems 278A Appendix 283A.1 Website 283A.2 Means, Variances, Covariances and Correlations 283A.3 Least Squares for Simple Regression 286A.4 Means and Variances of Least Squares Estimates 286A.5 Estimating E(𝑌 |𝑋) using a Smoother 288A.6 A Brief Introduction to Matrices and Vectors 290A.7 Random Vectors 295A.8 Least Squares Using Matrices 295A.9 The QR factorization 299A.10 Spectral Decomposition 300A.11 Maximum Likelihood Estimates 300A.12 The Box–Cox Method for Transformations 302A.13 Case Deletion in Linear Regression 305Bibliography 321Index 322