Examples and Problems in Mathematical Statistics

SHELEMYAHU ZACKS, PHD, is Distinguished Professor in the Department of Mathematical Sciences at Binghamton University. He has published several books and more than 170 journal articles on the design and analysis of experiments, statistical control of stochastic processes, statistical decision theory, sequential analysis, reliability, statistical methods in logistics, and sampling from finite populations. A Fellow of the American Statistical Association, Institute of Mathematical Sciences, and American Association for the Advancement of Sciences, Dr. Zacks is the author of Stage-Wise Adaptive Designs, also published by Wiley.

• Preface xvList of Random Variables xviiList of Abbreviations xix1 Basic Probability Theory 1Part I: Theory 11.1 Operations on Sets 11.2 Algebra and σ-Fields 21.3 Probability Spaces 41.4 Conditional Probabilities and Independence 61.5 Random Variables and Their Distributions 81.6 The Lebesgue and Stieltjes Integrals 121.6.1 General Definition of Expected Value: The Lebesgue Integral 121.6.2 The Stieltjes–Riemann Integral 171.6.3 Mixtures of Discrete and Absolutely Continuous Distributions 191.6.4 Quantiles of Distributions 191.6.5 Transformations 201.7 Joint Distributions Conditional Distributions and Independence 211.7.1 Joint Distributions 211.7.2 Conditional Expectations: General Definition 231.7.3 Independence 261.8 Moments and Related Functionals 261.9 Modes of Convergence 351.10 Weak Convergence 391.11 Laws of Large Numbers 411.11.1 The Weak Law of Large Numbers (WLLN) 411.11.2 The Strong Law of Large Numbers (SLLN) 421.12 Central Limit Theorem 441.13 Miscellaneous Results 471.13.1 Law of the Iterated Logarithm 481.13.2 Uniform Integrability 481.13.3 Inequalities 521.13.4 The Delta Method 531.13.5 The Symbols op and Op551.13.6 The Empirical Distribution and Sample Quantiles 55Part II: Examples 56Part III: Problems 73Part IV: Solutions to Selected Problems 932 Statistical Distributions 106Part I: Theory 1062.1 Introductory Remarks 1062.2 Families of Discrete Distributions 1062.2.1 Binomial Distributions 1062.2.2 Hypergeometric Distributions 1072.2.3 Poisson Distributions 1082.2.4 Geometric Pascal and Negative Binomial Distributions 1082.3 Some Families of Continuous Distributions 1092.3.1 Rectangular Distributions 1092.3.2 Beta Distributions 1112.3.3 Gamma Distributions 1112.3.4 Weibull and Extreme Value Distributions 1122.3.5 Normal Distributions 1132.3.6 Normal Approximations 1142.4 Transformations 1182.4.1 One-to-One Transformations of Several Variables 1182.4.2 Distribution of Sums 1182.4.3 Distribution of Ratios 1182.5 Variances and Covariances of Sample Moments 1202.6 Discrete Multivariate Distributions 1222.6.1 The Multinomial Distribution 1222.6.2 Multivariate Negative Binomial 1232.6.3 Multivariate Hypergeometric Distributions 1242.7 Multinormal Distributions 1252.7.1 Basic Theory 1252.7.2 Distribution of Subvectors and Distributions of Linear Forms 1272.7.3 Independence of Linear Forms 1292.8 Distributions of Symmetric Quadratic Forms of Normal Variables 1302.9 Independence of Linear and Quadratic Forms of Normal Variables 1322.10 The Order Statistics 1332.11 t-Distributions 1352.12 F-Distributions 1382.13 The Distribution of the Sample Correlation 1422.14 Exponential Type Families 1442.15 Approximating the Distribution of the Sample Mean: Edgeworth and Saddlepoint Approximations 1462.15.1 Edgeworth Expansion 1472.15.2 Saddlepoint Approximation 149Part II: Examples 150Part III: Problems 167Part IV: Solutions to Selected Problems 1813 Sufficient Statistics and the Information in Samples 191Part I: Theory 1913.1 Introduction 1913.2 Definition and Characterization of Sufficient Statistics 1923.2.1 Introductory Discussion 1923.2.2 Theoretical Formulation 1943.3 Likelihood Functions and Minimal Sufficient Statistics 2003.4 Sufficient Statistics and Exponential Type Families 2023.5 Sufficiency and Completeness 2033.6 Sufficiency and Ancillarity 2053.7 Information Functions and Sufficiency 2063.7.1 The Fisher Information 2063.7.2 The Kullback–Leibler Information 2103.8 The Fisher Information Matrix 2123.9 Sensitivity to Changes in Parameters 2143.9.1 The Hellinger Distance 214Part II: Examples 216Part III: Problems 230Part IV: Solutions to Selected Problems 2364 Testing Statistical Hypotheses 246Part I: Theory 2464.1 The General Framework 2464.2 The Neyman–Pearson Fundamental Lemma 2484.3 Testing One-Sided Composite Hypotheses in MLR Models 2514.4 Testing Two-Sided Hypotheses in One-Parameter Exponential Families 2544.5 Testing Composite Hypotheses with Nuisance Parameters—Unbiased Tests 2564.6 Likelihood Ratio Tests 2604.6.1 Testing in Normal Regression Theory 2614.6.2 Comparison of Normal Means: The Analysis of Variance 2654.7 The Analysis of Contingency Tables 2714.7.1 The Structure of Multi-Way Contingency Tables and the Statistical Model 2714.7.2 Testing the Significance of Association 2714.7.3 The Analysis of 2 × 2 Tables 2734.7.4 Likelihood Ratio Tests for Categorical Data 2744.8 Sequential Testing of Hypotheses 2754.8.1 The Wald Sequential Probability Ratio Test 276Part II: Examples 283Part III: Problems 298Part IV: Solutions to Selected Problems 3075 Statistical Estimation 321Part I: Theory 3215.1 General Discussion 3215.2 Unbiased Estimators 3225.2.1 General Definition and Example 3225.2.2 Minimum Variance Unbiased Estimators 3225.2.3 The Cramér–Rao Lower Bound for the One-Parameter Case 3235.2.4 Extension of the Cramér–Rao Inequality to Multiparameter Cases 3265.2.5 General Inequalities of the Cramér–Rao Type 3275.3 The Efficiency of Unbiased Estimators in Regular Cases 3285.4 Best Linear Unbiased and Least-Squares Estimators 3315.4.1 BLUEs of the Mean 3315.4.2 Least-Squares and BLUEs in Linear Models 3325.4.3 Best Linear Combinations of Order Statistics 3345.5 Stabilizing the LSE: Ridge Regressions 3355.6 Maximum Likelihood Estimators 3375.6.1 Definition and Examples 3375.6.2 MLEs in Exponential Type Families 3385.6.3 The Invariance Principle 3385.6.4 MLE of the Parameters of Tolerance Distributions 3395.7 Equivariant Estimators 3415.7.1 The Structure of Equivariant Estimators 3415.7.2 Minimum MSE Equivariant Estimators 3435.7.3 Minimum Risk Equivariant Estimators 3435.7.4 The Pitman Estimators 3445.8 Estimating Equations 3465.8.1 Moment-Equations Estimators 3465.8.2 General Theory of Estimating Functions 3475.9 Pretest Estimators 3495.10 Robust Estimation of the Location and Scale Parameters of Symmetric Distributions 349Part II: Examples 353Part III: Problems 381Part IV: Solutions of Selected Problems 3936 Confidence and Tolerance Intervals 406Part I: Theory 4066.1 General Introduction 4066.2 The Construction of Confidence Intervals 4076.3 Optimal Confidence Intervals 4086.4 Tolerance Intervals 4106.5 Distribution Free Confidence and Tolerance Intervals 4126.6 Simultaneous Confidence Intervals 4146.7 Two-Stage and Sequential Sampling for Fixed Width Confidence Intervals 417Part II: Examples 421Part III: Problems 429Part IV: Solution to Selected Problems 4337 Large Sample Theory for Estimation and Testing 439Part I: Theory 4397.1 Consistency of Estimators and Tests 4397.2 Consistency of the MLE 4407.3 Asymptotic Normality and Efficiency of Consistent Estimators 4427.4 Second-Order Efficiency of BAN Estimators 4447.5 Large Sample Confidence Intervals 4457.6 Edgeworth and Saddlepoint Approximations to the Distribution of the MLE: One-Parameter Canonical Exponential Families 4467.7 Large Sample Tests 4487.8 Pitman’s Asymptotic Efficiency of Tests 4497.9 Asymptotic Properties of Sample Quantiles 451Part II: Examples 454Part III: Problems 475Part IV: Solution of Selected Problems 4798 Bayesian Analysis in Testing and Estimation 485Part I: Theory 4858.1 The Bayesian Framework 4868.1.1 Prior Posterior and Predictive Distributions 4868.1.2 Noninformative and Improper Prior Distributions 4878.1.3 Risk Functions and Bayes Procedures 4898.2 Bayesian Testing of Hypothesis 4918.2.1 Testing Simple Hypothesis 4918.2.2 Testing Composite Hypotheses 4938.2.3 Bayes Sequential Testing of Hypotheses 4958.3 Bayesian Credibility and Prediction Intervals 5018.3.1 Credibility Intervals 5018.3.2 Prediction Intervals 5018.4 Bayesian Estimation 5028.4.1 General Discussion and Examples 5028.4.2 Hierarchical Models 5028.4.3 The Normal Dynamic Linear Model 5048.5 Approximation Methods 5068.5.1 Analytical Approximations 5068.5.2 Numerical Approximations 5088.6 Empirical Bayes Estimators 513Part II: Examples 514Part III: Problems 549Part IV: Solutions of Selected Problems 5579 Advanced Topics in Estimation Theory 563Part I: Theory 5639.1 Minimax Estimators 5639.2 Minimum Risk Equivariant Bayes Equivariant and Structural Estimators 5659.2.1 Formal Bayes Estimators for Invariant Priors 5669.2.2 Equivariant Estimators Based on Structural Distributions 5689.3 The Admissibility of Estimators 5709.3.1 Some Basic Results 5709.3.2 The Inadmissibility of Some Commonly Used Estimators 5759.3.3 Minimax and Admissible Estimators of the Location Parameter 5829.3.4 The Relationship of Empirical Bayes and Stein-Type Estimators of the Location Parameter in the Normal Case 584Part II: Examples 585Part III: Problems 592Part IV: Solutions of Selected Problems 596References 601Author Index 613Subject Index 617