Handbook of Statistical Data Editing and Imputation

Ton De Waal, PhD, is Head of the Department of Methodology at Statistics Netherlands, where he has also worked at the Division of Business Statistics. Dr. de Waal has written numerous papers in his areas of research interest, which include statistical data editing and imputation for business surveys and statistical disclosure control.Jeroen Pannekoek, PhD, is Senior Researcher in the Department of Methodology at Statistics Netherlands, where he currently leads the research program on data processing methodologies. He has published several papers on discrete data models, measurement errors, interviewer effects, and disclosure control methods.Sander Scholtus, MSc, is Researcher in the Department of Methodology at Statistics Netherlands. He has conducted extensive research on heuristic methods and algorithms for detecting and correcting errors in survey data.

• Preface ix1 Introduction to Statistical Data Editing and Imputation 11.1 Introduction 11.2 Statistical Data Editing and Imputation in the Statistical Process 41.3 Data, Errors, Missing Data, and Edits 61.4 Basic Methods for Statistical Data Editing and Imputation 131.5 An Edit and Imputation Strategy 17References 212 Methods for Deductive Correction 232.1 Introduction 232.2 Theory and Applications 242.3 Examples 272.4 Summary 55References 553 Automatic Editing of Continuous Data 573.1 Introduction 573.2 Automatic Error Localization of Random Errors 593.3 Aspects of the Fellegi–Holt Paradigm 633.4 Algorithms Based on the Fellegi–Holt Paradigm 653.5 Summary 1013.A Appendix: Chernikova’s Algorithm 103References 1044 Automatic Editing: Extensions to Categorical Data 1114.1 Introduction 1114.2 The Error Localization Problem for Mixed Data 1124.3 The Fellegi–Holt Approach 1154.4 A Branch-and-Bound Algorithm for Automatic Editing of Mixed Data 1294.5 The Nearest-Neighbor Imputation Methodology 140References 1585 Automatic Editing: Extensions to Integer Data 1615.1 Introduction 1615.2 An Illustration of the Error Localization Problem for Integer Data 1625.3 Fourier–Motzkin Elimination in Integer Data 1635.4 Error Localization in Categorical, Continuous, and Integer Data 1725.5 A Heuristic Procedure 1825.6 Computational Results 1835.7 Discussion 187References 1896 Selective Editing 1916.1 Introduction 1916.2 Historical Notes 1936.3 Micro-selection: The Score Function Approach 1956.4 Selection at the Macro-level 2086.5 Interactive Editing 2126.6 Summary and Conclusions 217References 2197 Imputation 2237.1 Introduction 2237.2 General Issues in Applying Imputation Methods 2267.3 Regression Imputation 2307.4 Ratio Imputation 2447.5 (Group) Mean Imputation 2467.6 Hot Deck Donor Imputation 2497.7 A General Imputation Model 2557.8 Imputation of Longitudinal Data 2617.9 Approaches to Variance Estimation with Imputed Data 2647.10 Fractional Imputation 271References 2728 Multivariate Imputation 2778.1 Introduction 2778.2 Multivariate Imputation Models 2808.3 Maximum Likelihood Estimation in the Presence of Missing Data 2858.4 Example: The Public Libraries 295References 2979 Imputation Under Edit Constraints 2999.1 Introduction 2999.2 Deductive Imputation 3019.3 The Ratio Hot Deck Method 3119.4 Imputing from a Dirichlet Distribution 3139.5 Imputing from a Singular Normal Distribution 3189.6 An Imputation Approach Based on Fourier–Motzkin Elimination 3349.7 A Sequential Regression Approach 3389.8 Calibrated Imputation of Numerical Data Under Linear Edit Restrictions 3439.9 Calibrated Hot Deck Imputation Subject to Edit Restrictions 349References 35810 Adjustment of Imputed Data 36110.1 Introduction 36110.2 Adjustment of Numerical Variables 36210.3 Adjustment of Mixed Continuous and Categorical Data 377References 38911 Practical Applications 39111.1 Introduction 39111.2 Automatic Editing of Environmental Costs 39111.3 The EUREDIT Project: An Evaluation Study 40011.4 Selective Editing in the Dutch Agricultural Census 420References 426Index 429