Modern Regression Methods

Thomas P. Ryan, PhD, served on the Editorial Review Board of the Journal of Quality Technology from 1990–2006, including three years as the book review editor. He is the author of four books, all of which are published by Wiley, and he is also an elected Fellow of the American Statistical Association, the American Society for Quality, and the Royal Statistical Society. A former consultant to Cytel Software Corporation, Dr. Ryan currently teaches advanced courses at statistics.com on the design of experiments, statistical process control, and engineering statistics.

• Preface. 1. Introduction.1.1 Simple Linear Regression Model.1.2 Uses of Regression Models.1.3 Graph the Data!1.4 Estimation of ß0 and ß1.1.5 Inferences from Regression Equations.1.6 Regression Through the Origin.1.7 Additional Examples.1.8 Correlation.1.9 Miscellaneous Uses of Regression.1.10 Fixed Versus Random Regressors.1.11 Missing Data.1.12 Spurious Relationships.1.13 Software.1.14 Summary.Appendix.References.Exercises.2. Diagnostics and Remedial Measures.2.1 Assumptions.2.2 Residual Plots.2.3 Transformations.2.4 Influential Observations.2.5 Outliers.2.6 Measurement Error.2.7 Software.2.8 Summary.Appendix.References.Exercises.3. Regression with Matrix Algebra.3.1 Introduction to Matrix Algebra.3.2 Matrix Algebra Applied to Regression.3.3 Summary.Appendix.References.Exercises.4. Introduction to Multiple Linear Regression.4.1 An Example of Multiple Linear Regression.4.2 Centering And Scaling.4.3 Interpreting Multiple Regression Coefficients.4.4 Indicator Variables.4.5 Separation or Not?4.6 Alternatives to Multiple Regression.4.7 Software.4.8 Summary.References.Exercises.5. Plots in Multiple Regression.5.1 Beyond Standardized Residual Plots.5.2 Some Examples.5.3 Which Plot?5.4 Recommendations.5.5 Partial Regression Plots.5.6 Other Plots For Detecting Influential Observations.5.7 Recent Contributions to Plots in Multiple Regression.5.8 Lurking Variables.5.9 Explanation of Two Data Sets Relative to R2.5.10 Software.5.11 Summary.References.Exercises.6. Transformations in Multiple Regression.6.1 Transforming Regressors.6.2 Transforming Y.6.3 Further Comments on the Normality Issue.6.4 Box-Cox Transformation.6.5 Box-Tidwell Revisited.6.6 Combined Box-Cox and Box-Tidwell Approach.6.7 Other Transformation Methods.6.8 Transformation Diagnostics.6.9 Software.6.10 Summary.References.Exercises.7. Selection of Regressors.7.1 Forward Selection.7.2 Backward Elimination.7.3 Stepwise Regression.7.4 All Possible Regressions.7.5 Newer Methods.7.6 Examples.7.7 Variable Selection for Nonlinear Terms.7.8 Must We Use a Subset?7.9 Model Validation.7.10 Software.7.11 Summary.Appendix.References.Exercises.8. Polynomial and Trigonometric Terms.8.1 Polynomial Terms.8.2 Polynomial-Trigonometric Regression.8.3 Software.8.4 Summary.References.Exercises.9. Logistic Regression.9.1 Introduction.9.2 One Regressor.9.3 A Simulated Example.9.4 Detecting Complete Separation, Quasicomplete Separation and Near Separation.9.5 Measuring the Worth of the Model.9.6 Determining the Worth of the Individual Regressors.9.7 Confidence Intervals.9.8 Exact Prediction.9.9 An Example With Real Data.9.10 An Example of Multiple Logistic Regression.9.11 Multicollinearity in Multiple Logistic Regression.9.12 Osteogenic Sarcoma Data Set.9.13 Missing Data.9.14 Sample Size Determination.9.15 Polytomous Logistic Regression.9.16 Logistic Regression Variations.9.17 Alternatives to Logistic Regression.9.18 Software for Logistic Regression.9.19 Summary.Appendix.References.Exercises.10. Nonparametric Regression.10.1 Relaxing Regression Assumptions.10.2 Monotone Regression.10.3 Smoothers.10.4 Variable Selection.10.5 Important Considerations in Smoothing.10.6 Sliced Inverse Regression.10.7 Projection Pursuit Regression.10.8 Software.10.9 Summary.Appendix.References.Exercises.11. Robust Regression.11.1 The Need for Robust Regression.11.2 Types of Outliers.11.3 Historical Development of Robust Regression.11.4 Goals of Robust Regression.11.5 Proposed High Breakdown Point Estimators.11.6 Approximating HBP Estimator Solutions.11.7 Other Methods for Detecting Multiple Outliers.11.8 Bounded Influence Estimators.11.9 Multistage Procedures.11.10 Other Robust Regression Estimators.11.11 Applications.11.12 Software for Robust Regression.11.13 Summary.References.Exercises.12. Ridge Regression.12.1 Introduction.12.2 How Do We Determine K?12.3 An Example.12.4 Ridge Regression for Prediction.12.5 Generalized Ridge Regression.12.6 Inferences in Ridge Regression.12.7 Some Practical Considerations.12.8 Robust Ridge Regression?12.9 Recent Developments in Ridge Regression.12.10 Other Biased Estimators.12.11 Software.12.12 Summary.Appendix.References.Exercises.13. Nonlinear Regression.13.1 Introduction.13.2 Linear Versus Nonlinear Regression.13.3 A Simple Nonlinear Example.13.4 Relative Offset Convergence Criterion.13.5 Adequacy of the Estimation Approach.13.6 Computational Considerations.13.7 Determining Model Adequacy.13.7.1 Lack-of-Fit Test.13.8 Inferences.13.9 An Application.13.10 Rational Functions.13.11 Robust Nonlinear Regression.13.12 Applications.13.13 Teaching Tools.13.14 Recent Developments.13.15 Software.13.16 Summary.Appendix.References.Exercises.14. Experimental Designs for Regression.14.1 Objectives for Experimental Designs.14.2 Equal Leverage Points.14.3 Other Desirable Properties of Experimental Designs.14.4 Model Misspecification.14.5 Range of Regressors.14.6 Algorithms for Design Construction.14.7 Designs for Polynomial Regression.14.8 Designs for Logistic Regression.14.9 Designs for Nonlinear Regression.14.10 Software.14.11 Summary.References.Exercises.15. Miscellaneous Topics in Regression.15.1 Piecewise Regression and Alternatives.15.2 Semiparametric Regression.15.3 Quantile Regression.15.4 Poisson Regression.15.5 Negative Binomial Regression.15.6 Cox Regression.15.7 Probit Regression.15.8 Censored Regression and Truncated Regression.15.8.1 Tobit Regression.15.9 Constrained Regression.15.10 Interval Regression.15.11 Random Coefficient Regression.15.12 Partial Least Squares Regression.15.13 Errors-in-Variables Regression.15.14 Regression with Life Data.15.15 Use of Regression in Survey Sampling.15.16 Bayesian Regression.15.17 Instrumental Variables Regression.15.18 Shrinkage Estimators.15.19 Meta-Regression.15.20 Classification and Regression Trees (CART).15.21 Multivariate Regression.References.Exercises.16. Analysis of Real Data Sets.16.1 Analyzing Buchanan’s Presidential Vote in Palm Beach County in 2000.16.2 Water Quality Data.16.3 Predicting Lifespan?16.4 Scottish Hill Races Data.16.5 Leukemia Data.16.6 Dosage Response Data.16.7 A Strategy for Analyzing Regression Data.16.8 Summary.References.Answers to Selected Exercises.Statistical Tables.Author Index.Subject Index.

"The book is to be praised in that it makes the reader aware of a large number of approaches to regression situations, and also to their possible pitfalls. It is thus an excellent basis for an experienced instructor to teach regression at different levels." (Springer, August 2010) "This book, at the undergraduate level and even at the graduate level, will be rewarding reading for anyone interested in learning the nuances of regression analysis." (Mathmatical Reviews, January 2010) "The exercises are interesting and thought-provoking throughout. If you liked the first edition, you will be pleased with this revision also." (International Statistical Review, August 2009)"The book is well written and has many exercises. It can serve as a very good textbook for scientists and engineers, with only basic statistics as a prerequisite. I also highly recommend it to practitioners who want to solve real-life prediction problems." (Computing Reviews, July 2009)"In this second edition, Ryan (author, editor, and educator) provides substantial updates and revisions of his popular text for statisticians to include new information on the most current advances and research in regression analysis" (SciTech Reviews, March 2009)"One would be hard-pressed to find another text that rivals this one in terms of coverage of the regression literature." (The American Statistician, 2009)"I strongly recommend the book as a reference for anyone teaching or using regression." (MAA Reviews, 2009)"Highly recommended for those already trained in mathematics and statistics who want a good guide to current practice and issues in multiple regression techniques." (Journal of Biopharmaceutical Statistics, 2009)