Statistics and the Evaluation of Evidence for Forensic Scientists

COLIN G. AITKEN, School of Mathematics, University of Edinburgh, UKFRANCO TARONI, School of Criminal Justice, University of Lausanne, Switzerland SILVIA BOZZA, Department of Economics, Ca’ Foscari University of Venice, Italy and School of Criminal Justice, University of Lausanne, Switzerland

• Foreword xviiPreface to Third Edition xxiPreface to Second Edition xxxPreface to First Edition xxxvii1 Uncertainty in Forensic Science 11.1 Introduction 11.2 Statistics and the Law 31.3 Uncertainty in Scientific Evidence 111.3.1 The Frequentist Method 151.3.2 Stains of Body Fluids 171.3.3 Glass Fragments 211.4 Terminology 291.5 Types of Data 341.6 Populations 361.7 Probability 411.7.1 Introduction 411.7.2 A Standard for Uncertainty 461.7.3 Events 551.7.4 Classical and Frequentist Definitions of Probability and Their Limitations 571.7.5 Subjective Definition of Probability 601.7.6 The Quantification of Probability Through a Betting Scheme 641.7.7 Probabilities and Frequencies: The Role of Exchangeability 691.7.8 Laws of Probability 781.7.9 Dependent Events and Background Information 821.7.10 Law of Total Probability 911.7.11 Updating of Probabilities 962 The Evaluation of Evidence 1012.1 Odds 1012.1.1 Complementary Events 1012.1.2 Examples 1042.1.3 Definition of Odds 1052.2 Bayes’ Theorem 1082.2.1 Statement of the Theorem 1092.2.2 Examples 1092.3 The Odds Form of Bayes’ Theorem 1212.3.1 Likelihood Ratio 1212.3.2 Bayes’ Factor and Likelihood Ratio 1252.3.3 Three-Way Tables 1302.3.4 Logarithm of the Likelihood Ratio 1342.4 The Value of Evidence 1382.4.1 Evaluation of Forensic Evidence 1382.4.2 Justification of the Use of the Likelihood Ratio 1542.4.3 Single Value for the Likelihood Ratio 1582.4.4 Role of Background Information 1612.4.5 Summary of Competing Propositions 1632.4.6 Qualitative Scale for the Value of the Evidence 1682.5 Errors in Interpretation 1802.5.1 Fallacy of the Transposed Conditional 1862.5.2 Source Probability Error 1902.5.3 Ultimate Issue Error 1942.5.4 Defence Attorney’s Fallacy 1942.5.5 Probability (Another Match) Error 1962.5.6 Numerical Conversion Error 1992.5.7 False Positive Fallacy 2022.5.8 Expected Value Fallacy 2032.5.9 Uniqueness 2062.5.10 Other Difficulties 2092.5.11 Empirical Evidence of Errors in Interpretation 2202.6 Misinterpretations 2332.7 Explanation of Transposed Conditional, Defence Attorney’s and False Positive Fallacies 2362.7.1 Explanation of the Fallacy of the Transposed Conditional 2362.7.2 Explanation of the Defence Attorney’s Fallacy 2392.7.3 Explanation of the False Positive Fallacy 2412.8 Making Coherent Decisions 2452.8.1 Elements of Statistical Decision Theory 2462.8.2 Decision Analysis: An Example 2492.9 Graphical Probabilistic Models: Bayesian Networks 2542.9.1 Elements of the Bayesian Networks 2562.9.2 The Construction of Bayesian Networks 2612.9.3 Bayesian Decision Networks (Influence Diagrams) 2723 Historical Review 2793.1 Early History 2793.2 The Dreyfus Case 2863.3 Statistical Arguments by Early Twentieth- Century Forensic Scientists 2933.4 People v.Collins 2993.5 Discriminating Power 3073.5.1 Derivation 3073.5.2 Evaluation of Evidence by Discriminating Power 3103.5.3 Finite Samples 3163.5.4 Combination of Independent Systems 3193.5.5 Correlated Attributes 3213.6 Significance Probabilities 3253.6.1 Calculation of Significance Probabilities 3263.6.2 Relationship to Likelihood Ratio 3333.6.3 Combination of Significance Probabilities 3383.7 Coincidence Probabilities 3423.7.1 Introduction 3423.7.2 Comparison Stage 3463.7.3 Significance Stage 3473.8 Likelihood Ratio 3514 Bayesian Inference 3594.1 Introduction 3594.2 Inference for a Proportion 3684.2.1 Interval Estimation 3744.2.2 Estimation with Zero Occurrences in a Sample 3814.2.3 Uncertainty on Sensitivity and Specificity 3874.3 Sampling 3924.3.1 Choice of Sample Size in Large Consignments 3984.3.2 Choice of Sample Size in Small Consignments 4134.4 Bayesian Networks for Sampling Inspection 4204.4.1 Large Consignments 4204.4.2 Small Consignments 4254.5 Inference for a Normal Mean 4294.5.1 Known Variance 4314.5.2 Unknown Variance 4384.5.3 Interval Estimation 4454.6 Quantity Estimation 4494.6.1 Predictive Approach in Small Consignments 4524.6.2 Predictive Approach in Large Consignments 4614.7 Decision Analysis 4644.7.1 Standard Loss Functions 4654.7.2 Decision Analysis for Forensic Sampling 4715 Evidence and Propositions: Theory 4835.1 The Choice of Propositions and Pre-Assessment 4835.2 Levels of Propositions and Roles of the Forensic Scientist 4855.3 The Formal Development of a Likelihood Ratio for Different Propositions and Discrete Characteristics 4995.3.1 Likelihood Ratio with Source Level Propositions 4995.3.2 Likelihood Ratio with Activity Level Propositions 5195.3.3 Likelihood Ratio with Offence Level Propositions 5535.4 Validation of Bayesian Network Structures: An Example 5625.5 Pre-Assessment 5685.5.1 Pre-assessment of the Case 5685.5.2 Pre-assessment of Evidence 5755.5.3 Pre-assessment: A Practical Example 5765.6 Combination of Items of Evidence 5925.6.1 A Difficulty in Combining Evidence: The Problem of Conjunction 5945.6.2 Generic Patterns of Inference in Combining Evidence 5986 Evidence and Propositions: Practice 6156.1 Examples for Evaluation given Source Level Propositions 6156.1.1 General Population 6166.1.2 Particular Population 6176.1.3 A Note on The Appropriate Databases for Evaluation Given Source Level Propositions 6196.1.4 Two Trace Problem 6276.1.5 Many Samples 6336.1.6 Multiple Propositions 6376.1.7 A Note on Biological Traces 6546.1.8 Additional Considerations on Source Level Propositions 6706.2 Examples for Evaluation given Activity Level Propositions 6996.2.1 A Practical Approach to Fibres Evaluation 7016.2.2 A Practical Approach to Glass Evaluation 7046.2.3 The Assignment of Probabilities for Transfer Events 7136.2.4 The Assignment of Probabilities for Background Traces 7346.2.5 Presence of Material with Non-corresponding Features 7396.2.6 Absence of Evidence for Activity Level Propositions 7416.3 Examples for Evaluation given Offence Level Propositions 7456.3.1 One Stain, k Offenders 7456.3.2 Two Stains, One Offender 7526.3.3 Paternity and The Combination of Likelihood Ratios 7566.3.4 Probability of Paternity 7626.3.5 Absence of Evidence for Offence Level Propositions 7686.3.6 A Note on Relevance and Offence Level Propositions 7736.4 Summary 7746.4.1 Stain Known to Have Been Left by Offenders: Source-Level Propositions 7746.4.2 Material Known to Have Been (or Not to Have Been) Left by Offenders: Activity-Level Propositions 7776.4.3 Stain May Not Have Been Left by Offenders: Offence-Level Propositions 7797 Data Analysis 7837.1 Introduction 7837.2 Theory for Discrete Data 7857.2.1 Data of Independent Counts with a Poisson Distribution 7877.2.2 Data of Independent Counts with a Binomial Distribution 7917.2.3 Data of Independent Counts with a Multinomial Distribution 7937.3 Theory for Continuous Univariate Data 7987.3.1 Assessment of Similarity Only 8027.3.2 Sources of Variation: Two-Level Models 8087.3.3 Transfer Probability 8107.4 Normal Between-Source Variation 8147.4.1 Marginal Distribution of Measurements 8147.4.2 Approximate Derivation of the Likelihood Ratio 8177.4.3 Lindley’s Approach 8207.4.4 Interpretation of Result 8257.4.5 Examples 8277.5 Non-normal Between-Source Variation 8307.5.1 Estimation of a Probability Density Function 8317.5.2 Kernel Density Estimation for Between-Source Data 8427.5.3 Examples 8447.6 Multivariate Analysis 8497.6.1 Introduction 8497.6.2 Multivariate Two-Level Models 8517.6.3 A Note on Sensitivity 8647.6.4 Case Study for Two-Level Data 8657.6.5 Three-Level Models 8767.7 Discrimination 8827.7.1 Discrete Data 8847.7.2 Continuous Data 8897.7.3 Autocorrelated Data 8937.7.4 Multivariate Data 8947.7.5 Cut-Offs and Legal Thresholds 8997.8 Score-Based Models 9067.8.1 Example 9107.9 Bayes’ Factor and Likelihood Ratio (cont.) 9138 Assessment of the Performance of Methods for the Evaluation of Evidence 9198.1 Introduction 9198.2 Properties of Methods for Evaluation 9288.3 General Topics Relating to Sample Size Estimation and to Assessment 9338.3.1 Probability of Strong Misleading Evidence: A Sample Size Problem 9338.3.2 Calibration 9488.4 Assessment of Performance of a Procedure for the Calculation of the Likelihood Ratio 9528.4.1 Histograms and Tippett Plots 9568.4.2 False Positive Rates, False Negative Rates and DET Plots 9598.4.3 Empirical Cross-Entropy 9618.5 Case Study: Kinship Analysis 9728.6 Conclusion 979Appendix A Probability Distributions 981A.1 Introduction 981A.2 Probability Distributions for Counts 988A.2.1 Probabilities 988A.2.2 Summary Measures 990A.2.3 Binomial Distribution 995A.2.4 Multinomial Distribution 997A.2.5 Hypergeometric Distribution 998A.2.6 Poisson Distribution 1000A.2.7 Beta-Binomial and Dirichlet-Multinomial Distributions 1002A.3 Measurements 1005A.3.1 Summary Statistics 1005A.3.2 Normal Distribution 1007A.3.3 Jeffreys’ Prior Distributions 1021A.3.4 Student’s t-Distribution 1021A.3.5 Gamma and Chi-Squared Distributions 1025A.3.6 Inverse Gamma and Inverse Chi-Squared Distributions 1026A.3.7 Beta Distribution 1028A.3.8 Dirichlet Distribution 1032A.3.9 Multivariate Normal Distribution and Correlation 1035A.3.10 Wishart Distribution 1040A.3.11 Inverse Wishart Distribution 1041Appendix B Matrix Properties 1043B.1 Matrix Terminology 1043B.1.1 The Trace of a Square Matrix 1044B.1.2 The Transpose of a Matrix 1044B.1.3 Addition of Two Matrices 1045B.1.4 Determinant of a Matrix 1045B.1.5 Matrix Multiplication 1046B.1.6 The Inverse of a Matrix 1048B.1.7 Completion of the Square 1049References 1051Notation 1143Cases 1157Author Index 1163Subject Index 1187

"I hope that every forensic laboratory in the United States and Europe has a copy of this book, and that they rapidly wear out from repeated use. It is a tremendous resource. It could also be a valuable textbook for a methods course in criminology departments."—David Banks, Department of Statistical Science, Duke University, Durham, NC, USA