Probability and Conditional Expectation

Rolf Steyer,Institute of Psychology, University of Jena, Germany Werner Nagel,Institute of Mathematics, University of Jena, Germany

• Part I Measure-Theoretical Foundations of Probability Theory1 Measure 31.1 Introductory Examples 31.2 σ-Algebra and Measurable Space 41.2.1 σ-Algebra Generated by a Set System 91.2.2 σ-Algebra of Borel Sets on Rn 121.2.3 σ-Algebra on a Cartesian Product 131.2.4 ∩-Stable Set Systems That Generate a σ-Algebra 151.3 Measure and Measure Space 161.3.1 σ-Additivity and Related Properties 171.3.2 Other Properties 181.4 Specific Measures 201.4.1 Dirac Measure and Counting Measure 211.4.2 Lebesgue Measure 221.4.3 Other Examples of a Measure 231.4.4 Finite and σ-Finite Measures 231.4.5 Product Measure 241.5 Continuity of a Measure 251.6 Specifying a Measure via a Generating System 271.7 σ-Algebra That is Trivial With Respect to a Measure 281.8 Proofs 281.9 Exercises 312 Measurable Mapping 412.1 Image and Inverse Image 412.2 Introductory Examples 422.2.1 Example 1: Rectangles 422.2.2 Example 2: Flipping two Coins 442.3 Measurable Mapping 462.3.1 Measurable Mapping 462.3.2 σ-Algebra Generated by a Mapping 512.3.3 Final σ-Algebra 542.3.4 Multivariate Mapping 542.3.5 Projection Mapping 562.3.6 Measurability With Respect to a Mapping 562.4 Theorems on Measurable Mappings 582.4.1 Measurability of a Composition 592.4.2 Theorems on Measurable Functions 612.5 Equivalence of Two Mappings With Respect to a Measure 642.6 Image Measure 672.7 Proofs 702.8 Exercises 753 Integral 833.1 Definition 833.1.1 Integral of a Nonnegative Step Function 833.1.2 Integral of a Nonnegative Measurable Function 883.1.3 Integral of a Measurable Function 933.2 Properties 963.2.1 Integral of μ-Equivalent Functions 983.2.2 Integral With Respect to a Weighted Sum of Measures 1003.2.3 Integral With Respect to an Image Measure 1023.2.4 Convergence Theorems 1033.3 Lebesgue and Riemann Integral 1043.4 Density 1063.5 Absolute Continuity and the Radon-Nikodym Theorem 1083.6 Integral With Respect to a Product Measure 1103.7 Proofs 1113.8 Exercises 120Part II Probability, Random Variable and its Distribution4 Probability Measure 1274.1 Probability Measure and Probability Space 1274.1.1 Definition 1274.1.2 Formal and Substantive Meaning of Probabilistic Terms 1284.1.3 Properties of a Probability Measure 1284.1.4 Examples 1304.2 Conditional Probability 1324.2.1 Definition 1324.2.2 Filtration and Time Order Between Events and Sets of Events 1334.2.3 Multiplication Rule 1354.2.4 Examples 1364.2.5 Theorem of Total Probability 1374.2.6 Bayes’ Theorem 1384.2.7 Conditional-Probability Measure 1394.3 Independence 1434.3.1 Independence of Events 1434.3.2 Independence of Set Systems 1444.4 Conditional Independence Given an Event 1454.4.1 Conditional Independence of Events Given an Event 1454.4.2 Conditional Independence of Set Systems Given an Event 1464.5 Proofs 1484.6 Exercises 1505 Random Variable, Distribution, Density, and Distribution Function 1555.1 Random Variable and its Distribution 1555.2 Equivalence of Two Random Variables With Respect to a Probability Measure 1615.2.1 Identical and P-Equivalent Random Variables 1615.2.2 P-Equivalence, PB-Equivalence, and Absolute Continuity 1645.3 Multivariate Random Variable 1675.4 Independence of Random Variables 1695.5 Probability Function of a Discrete Random Variable 1755.6 Probability Density With Respect to a Measure 1785.6.1 General Concepts and Properties 1785.6.2 Density of a Discrete Random Variable 1805.6.3 Density of a Bivariate Random Variable 1805.7 Uni- or Multivariate Real-Valued Random Variable 1825.7.1 Distribution Function of a Univariate Real-Valued Random Variable 1825.7.2 Distribution Function of a Multivariate Real-Valued Random Variable 1845.7.3 Density of a Continuous Univariate Real-Valued Random Variable 1855.7.4 Density of a Continuous Multivariate Real-Valued Random Variable 1875.8 Proofs 1885.9 Exercises 1966 Expectation, Variance, and Other Moments 1996.1 Expectation 1996.1.1 Definition 1996.1.2 Expectation of a Discrete Random Variable 2006.1.3 Computing the Expectation Using a Density 2026.1.4 Transformation Theorem 2036.1.5 Rules of Computation 2066.2 Moments, Variance, and Standard Deviation 2076.3 Proofs 2126.4 Exercises 2137 Linear Quasi-Regression, Covariance, and Correlation 2177.1 Linear Quasi-Regression 2177.2 Covariance 2207.3 Correlation 2247.4 Expectation Vector and Covariance Matrix 2277.4.1 Random Vector and Random Matrix 2277.4.2 Expectation of a Random Vector and a Random Matrix 2287.4.3 Covariance Matrix of two Multivariate Random Variables 2297.5 Multiple Linear Quasi-Regression 2317.6 Proofs 2337.7 Exercises 2378 Some Distributions 2458.1 Some Distributions of Discrete Random Variables 2458.1.1 Discrete Uniform Distribution 2458.1.2 Bernoulli Distribution 2468.1.3 Binomial Distribution 2478.1.4 Poisson Distribution 2508.1.5 Geometric Distribution 2528.2 Some Distributions of Continuous Random Variables 2548.2.1 Continuous Uniform Distribution 2548.2.2 Normal Distribution 2568.2.3 Multivariate Normal Distribution 2598.2.4 Central χ2-Distribution 2628.2.5 Central t -Distribution 2648.2.6 Central F-Distribution 2668.3 Proofs 2678.4 Exercises 271Part III Conditional Expectation and Regression9 Conditional Expectation Value and Discrete Conditional Expectation 2779.1 Conditional Expectation Value 2779.2 Transformation Theorem 2809.3 Other Properties 2829.4 Discrete Conditional Expectation 2839.5 Discrete Regression 2859.6 Examples 2879.7 Proofs 2919.8 Exercises 29110 Conditional Expectation 29510.1 Assumptions and Definitions 29510.2 Existence and Uniqueness 29710.2.1 Uniqueness With Respect to a Probability Measure 29810.2.2 A Necessary and Sufficient Condition of Uniqueness 29910.2.3 Examples 30010.3 Rules of Computation and Other Properties 30110.3.1 Rules of Computation 30110.3.2 Monotonicity 30210.3.3 Convergence Theorems 30210.4 Factorization, Regression, and Conditional Expectation Value 30610.4.1 Existence of a Factorization 30610.4.2 Conditional Expectation and Mean-Squared Error 30710.4.3 Uniqueness of a Factorization 30810.4.4 Conditional Expectation Value 30910.5 Characterizing a Conditional Expectation by the Joint Distribution 31210.6 Conditional Mean Independence 31310.7 Proofs 31810.8 Exercises 32111 Residual, Conditional Variance, and Conditional Covariance 32911.1 Residual With Respect to a Conditional Expectation 32911.2 Coefficient of Determination and Multiple Correlation 33311.3 Conditional Variance and Covariance Given a σ-Algebra 33811.4 Conditional Variance and Covariance Given a Value of a Random Variable 33911.5 Properties of Conditional Variances and Covariances 34211.6 Partial Correlation 34511.7 Proofs 34711.8 Exercises 34812 Linear Regression 35712.1 Basic Ideas 35712.2 Assumptions and Definitions 35912.3 Examples 36112.4 Linear Quasi-Regression 36612.5 Uniqueness and Identification of Regression Coefficients 36712.6 Linear Regression 36912.7 Parametrizations of a Discrete Conditional Expectation 37012.8 Invariance of Regression Coefficients 37412.9 Proofs 37512.10Exercises 37713 Linear Logistic Regression 38113.1 Logit Transformation of a Conditional Probability 38113.2 Linear Logistic Parametrization 38313.3 A Parametrization of a Discrete Conditional Probability 38513.4 Identification of Coefficients of a Linear Logistic Parametrization 38713.5 Linear Logistic Regression and Linear Logit Regression 38813.6 Proofs 39413.7 Exercises 39614 Conditional Expectation With Respect to a Conditional-Probability Measure 39914.1 Introductory Examples 39914.2 Assumptions and Definitions 40414.3 Properties 41014.4 Partial Conditional Expectation 41214.5 Factorization 41314.5.1 Conditional Expectation Value With Respect to PB 41414.5.2 Uniqueness of Factorizations 41514.6 Uniqueness 41514.6.1 A Necessary and Sufficient Condition of Uniqueness 41514.6.2 Uniqueness w.r.t. P and Other Probability Measures 41714.6.3 Necessary and Sufficient Conditions of P-Uniqueness 41814.6.4 Properties Related to P-Uniqueness 42014.7 Conditional Mean Independence With Respect to PZ=z 42414.8 Proofs 42614.9 Exercises 43115 Conditional Effect Functions of a Discrete Regressor 43715.1 Assumptions and Definitions 43715.2 Conditional Intercept Function and Effect Functions 43815.3 Implications of Independence of X and Z for Regression Coefficients 44115.4 Adjusted Conditional Effect Functions 44315.5 Conditional Logit Effect Functions 44715.6 Implications of Independence of X and Z for the Logit Regression Coefficients 45015.7 Proofs 45215.8 Exercises 454Part IV Conditional Independence and Conditional Distribution16 Conditional Independence 45916.1 Assumptions and Definitions 45916.1.1 Two Events 45916.1.2 Two Sets of Events 46116.1.3 Two Random Variables 46216.2 Properties 46316.3 Conditional Independence and Conditional Mean Independence 47016.4 Families of Events 47316.5 Families of Set Systems 47316.6 Families of Random Variables 47516.7 Proofs 47816.8 Exercises 48617 Conditional Distribution 49117.1 Conditional Distribution Given a σ-Algebra or a Random Variable 49117.2 Conditional Distribution Given a Value of a Random Variable 49417.3 Existence and Uniqueness 49717.3.1 Existence 49717.3.2 Uniqueness of the Functions PY |C ( ·, A′) 49817.3.3 Common Null Set (CNS) Uniqueness of a Conditional Distribution 49917.4 Conditional-Probability Measure Given a Value of a Random Variable 50217.5 Decomposing the Joint Distribution of Random Variables 50417.6 Conditional Independence and Conditional Distributions 50617.7 Expectations With Respect to a Conditional Distribution 51117.8 Conditional Distribution Function and Probability Density 51317.9 Conditional Distribution and Radon-Nikodym Density 51617.10Proofs 52017.11Exercises 536References 541