Survey Data Harmonization in the Social Sciences

Irina Tomescu-Dubrow is Professor of Sociology at the Institute of Philosophy and Sociology at the Polish Academy of Sciences (PAN), and director of the Graduate School for Social Research at PAN.Christof Wolf is President of GESIS — Leibniz Institute for the Social Sciences and Professor of Sociology at the University of Mannheim in Germany.Kazimierz M. Slomczynski is Professor of Sociology at the Institute of Philosophy and Sociology, the Polish Academy of Sciences (IFiS PAN) and Academy Professor of Sociology at the Ohio State University (OSU). He co-directs CONSIRT - the Cross-national Studies: Interdisciplinary Research and Training program at OSU and IFiS PAN.J. Craig Jenkins is Academy Professor of Sociology and Senior Research Scientist at the Mershon Center for International Security at the Ohio State University.

• Preface and Acknowledgments xvAbout the Editors xviiAbout the Contributors xviii1 Objectives and Challenges of Survey Data Harmonization 1Kazimierz M. Slomczynski, Irina Tomescu-Dubrow, J. Craig Jenkins, and Christof Wolf1.1 Introduction 11.2 What is the Harmonization of Survey Data? 21.2.1 Ex-ante, Input and Output, Survey Harmonization 31.3 Why Harmonize Social Survey Data? 51.3.1 Comparison and Equivalence 61.4 Harmonizing Survey Data Across and Within Countries 71.4.1 Harmonizing Across Countries 71.4.2 Harmonizing Within the Country 81.5 Sources of Knowledge for Survey Data Harmonization 81.6 Challenges to Survey Harmonization 91.6.1 Population Representation (Sampling Design) 101.6.2 Instruments and Their Adaptation (Including Translation) 101.6.3 Preparation for Interviewing (Including Pretesting) 111.6.4 Fieldwork (Including Modes of Interviewing) 111.6.5 Data Preparation (Including Building Data Files) 121.6.6 Data Processing, Quality Controls, and Adjustments 121.6.7 Data Dissemination 131.7 Survey Harmonization and Standardization Processes 131.8 Quality of the Input and the End-product of Survey Harmonization 141.9 Relevance of Harmonization Methodology to the FAIR Data Principles 151.10 Ethical and Legal Issues 151.11 How to Read this Volume? 16References 172 The Effects of Data Harmonization on the Survey Research Process 21Ranjit K. Singh, Arnim Bleier, and Peter Granda2.1 Introduction 212.2 Part 1: Harmonization: Origins and Relation to Standardization 222.2.1 Early Conceptions of Standardization and Harmonization 222.2.2 Foundational Work of International Survey Programs 232.2.3 The Growing Impact of Data Harmonization 232.3 Part 2: Stakeholders and Division of Labor 252.3.1 Stakeholders 262.3.1.1 International Actors and Funding Agencies 262.3.1.2 Data Producers 262.3.1.3 Archives 272.3.1.4 Data Users 272.3.2 Toward an Integrative View on Harmonization 282.3.2.1 Harmonization Cost 292.3.2.2 Harmonization Quality 292.3.2.3 Harmonization Fit 302.3.2.4 Moving Forward 302.4 Part 3: New Data Types, New Challenges 312.4.1 Designed Data and Organic Data 312.4.2 Stakeholders in the Collection of Organic Data 322.4.2.1 Producers 322.4.2.2 Archives 322.4.2.3 Users 332.4.2.4 Harmonization of Organic Data 332.5 Conclusion 33References 35Part I Ex-ante harmonization of survey instruments and non-survey data 393 Harmonization in the World Values Survey 41Kseniya Kizilova, Jaime Diez-Medrano, Christian Welzel, and Christian Haerpfer3.1 Introduction 413.2 Applied Harmonization Methods 423.3 Documentation and Quality Assurance 483.4 Challenges to Harmonization 493.5 Software Tools 513.6 Recommendations 52References 544 Harmonization in the Afrobarometer 57Carolyn Logan, Robert Mattes, and Francis Kibirige4.1 Introduction 574.2 Core Principles 584.3 Applied Harmonization Methods 604.3.1 Sampling 604.3.2 Training 614.3.3 Fieldwork and Data Collection 624.3.4 Questionnaire 624.3.5 Translation 644.3.6 Data Management 654.3.7 Documentation 654.4 Harmonization and Country Selection 664.5 Software Tools and Harmonization 664.6 Challenges to Harmonization 674.6.1 Local Knowledge, Flexibility/Adaptability, and the “Dictatorship of Harmonization” 684.6.2 The Quality-Cost Trade-off and Implications for Harmonization 684.6.3 Final Challenge: “Events” 694.7 Recommendations 70References 715 Harmonization in the National Longitudinal Surveys of Youth (NLSY) 73Elizabeth Cooksey, Rosella Gardecki, Carole Lunney, and Amanda Roose5.1 Introduction 735.2 Cross-Cohort Design 755.3 Applied Harmonization 765.4 Challenges to Harmonization 805.5 Documentation and Quality Assurance 825.6 Software Tools 845.7 Recommendations and Some Concluding Thoughts 86References 876 Harmonization in the Comparative Study of Electoral Systems (CSES) Projects 89Stephen Quinlan, Christian Schimpf, Katharina Blinzler, and Slaven Zivkovic6.1 Introducing the CSES 896.2 Harmonization Principles and Technical Infrastructure 916.3 Ex-ante Input Harmonization 916.3.1 Module Questionnaire 926.3.2 Macro Data 946.4 Ex-ante Output Harmonization 976.4.1 Demographic Variables in CSES Modules 976.4.2 Harmonizing Party Data in Modules 986.4.3 Derivative Variables 996.5 Exploring Interplay Between Ex-ante and Ex-post Harmonization 1016.5.1 Demographic Variables in CSES IMD 1016.5.2 Harmonizing Party Data in CSES IMD 1026.6 Taking Stock and New Frontiers in Harmonization 104References 1057 Harmonization in the East Asian Social Survey 107Noriko Iwai, Tetsuo Mo, Jibum Kim, Chyi-In Wu, and Weidong Wang7.1 Introduction 1077.2 Characteristics of the EASS and its Harmonization Process 1087.2.1 Outline of the East Asian Social Survey 1087.2.2 Harmonization Process of the EASS 1117.2.2.1 Establishing the Module Theme 1117.2.2.2 Selecting Subtopics and Questions 1127.2.2.3 Harmonization of Standard Background Variables 1137.2.2.4 Harmonization of Answer Choices and Scales 1147.2.2.5 Translation of Questions and Answer Choices 1157.3 Documentation and Quality Assurance 1157.3.1 Five Steps to Harmonize the EASS Integrated Data 1157.3.2 Documentation of the EASS Integrated Data 1177.4 Challenges to Harmonization 1187.4.1 How to Translate “Fair” and Restriction by Copyright 1187.4.2 Difficulty in Synchronizing the Data Collection Phase 1217.5 Software Tools 1227.6 Recommendations 122Acknowledgment 123References 1238 Ex-ante Harmonization of Official Statistics in Africa (SHaSA) 125Dossina YeoAbbreviations 1258.1 Introduction 1278.2 Applied Harmonization Methods 1288.2.1 Examples of Ex-ante Harmonization Methods: The Cases of GPS Data and CRVS 1318.2.1.1 Governance, Peace and Security (GPS) Statistics Initiative 1318.2.1.2 Development of Civil Registration and Vital Statistics (CRVS) 1328.2.2 Examples of Ex-post Harmonization: The Cases of Labor Statistics, ATSY, ASY and KeyStats, and ICP-Africa Program 1328.3 Quality Assurance Framework 1348.4 Challenges to Statistical Harmonization in Africa 1368.4.1 Challenges to the Implementation of NSDS 1378.4.2 Challenges with Ex-ante Harmonization: Examples of GPS and ICP Initiatives 1388.4.3 Challenges with Ex-post Harmonization: Examples of KeyStats and ATSY 1398.5 Common Software Tools Used 1398.6 Conclusion and Recommendations 140References 142Part II Ex-post harmonization of national social surveys 1459 Harmonization for Cross-National Secondary Analysis: Survey Data Recycling 147Irina Tomescu-Dubrow, Kazimierz M. Slomczynski, Ilona Wysmulek, Przemek Powałko, Olga Li, Yamei Tu, Marcin Slarzynski, Marcin W. Zielinski, and Denys Lavryk9.1 Introduction 1479.2 Harmonization Methods in the SDR Project 1499.2.1 Building the Harmonized SDR2 Database 1509.3 Documentation and Quality Assurance 1559.4 Challenges to Harmonization 1569.5 Software Tools of the SDR Project 1619.5.1 The SDR Portal 1619.5.2 The SDR2 COTTON FILE 1629.6 Recommendations 1629.6.1 Recommendations for Researchers Interested in Harmonizing Survey Data Ex-Post 1629.6.2 Recommendations for SDR2 Users 163Acknowledgments 164References 1649.A Data Quality Indicators in SDR2 16610 Harmonization of Panel Surveys: The Cross-National Equivalent File 169Dean R. Lillard10.1 Introduction 16910.2 Applied Harmonization Methods 17010.2.1 CNEF Country Data Sources, Current and Planned 17610.3 Current CNEF Partners 17610.3.1 The HILDA Survey 17610.3.2 The SLID 17610.3.3 The CFPS 17710.3.4 The SOEP 17710.3.4.1 The BHPS 17710.3.4.2 Understanding Society, UKHLS 17810.3.5 The ITA.LI 17810.3.6 The JHPS 17810.3.7 The RLMS-HSE 17810.3.8 The KLIPS 17910.3.9 The Swedish Pseudo-Panel 17910.3.10 The SHP 17910.3.11 The PSID 17910.4 Planned CNEF Partners 18010.4.1 The ASEP 18010.4.2 LISA 18010.4.3 The ILS 18010.4.4 The MxFLS 18010.4.5 The NIDS 18110.4.6 The PSFD 18110.5 Documentation and Quality Assurance 18110.6 Challenges to Harmonization 18310.7 Recommendations for Researchers Interested in Harmonizing Panel Survey Data 18510.8 Conclusion 186References 18711 Harmonization of Survey Data from UK Longitudinal Studies: CLOSER 189Dara O’Neill and Rebecca Hardy11.1 Introduction 18911.2 Applied Harmonization Methods 19111.2.1 Occupational Social Class 19111.2.2 Body Size/Anthropometric Data 19311.2.3 Mental Health 19411.2.4 Harmonization Methods: Divergence and Convergence 19511.3 Documentation and Quality Assurance 19611.4 Challenges to Harmonization 19811.5 Software Tools 19911.6 Recommendations 200Acknowledgments 202References 20212 Harmonization of Census Data: IPUMS – International 207Steven Ruggles, Lara Cleveland, and Matthew Sobek12.1 Introduction 20712.2 Project History 20812.2.1 Evolution of the Web Dissemination System 21012.3 Applied Harmonization Methods 21012.4 Documentation and Quality Assurance 21512.5 Challenges to Harmonization 21712.6 Software Tools 22112.6.1 Metadata Tools 22112.6.2 Data Reformatting 22112.6.3 Data Harmonization 22112.6.4 Dissemination System 22212.7 Team Organization and Project Management 22212.8 Lessons and Recommendations 223References 225Part III Domain-driven ex-post harmonization 22713 Maelstrom Research Approaches to Retrospective Harmonization of Cohort Data for Epidemiological Research 229Tina W. Wey and Isabel Fortier13.1 Introduction 22913.2 Applied Harmonization Methods 23013.2.1 Implementing the Project 23313.2.1.1 Initiating Activities and Organizing the Operational Framework 23313.2.1.2 Assembling Study Information and Selecting Final Participating Studies (Guidelines Step 1) 23413.2.1.3 Defining Target Variables to be Harmonized (the DataSchema) and Evaluating Harmonization Potential across Studies (Guidelines Step 2) 23513.2.2 Producing the Harmonized Datasets 23613.2.2.1 Processing Data (Guidelines Step 3a) 23613.2.2.2 Processing Study-Specific Data to Generate Harmonized Datasets (Guidelines Step 3b) 23713.3 Documentation and Quality Assurance 23813.4 Challenges to Harmonization 24013.5 Software Tools 24113.6 Recommendations 243Acknowledgments 244References 24514 Harmonizing and Synthesizing Partnership Histories from Different German Survey Infrastructures 249Bernd Weiß, Sonja Schulz, Lisa Schmid, Sebastian Sterl, and Anna-Carolina Haensch14.1 Introduction 24914.2 Applied Harmonization Methods 25014.2.1 Data Search Strategy and Data Access 25014.2.2 Processing and Harmonizing Data 25314.2.2.1 Harmonizing Partnership Biography Data 25314.2.2.2 Harmonizing Additional Variables on Respondents’ or Couples’ Characteristics 25414.3 Documentation and Quality Assurance 25514.3.1 Documentation 25514.3.2 Quality Assurance 25614.3.2.1 Process-Related Quality Assurance 25614.3.2.2 Benchmarking the Harmonized HaSpaD Data Set with Official Statistics 25614.4 Challenges to Harmonization 25814.4.1 Analyzing Harmonized Complex Survey Data 25814.4.2 Sporadically and Systematically Missing Data 25914.5 Software Tools 26014.6 Recommendations 26214.6.1 Harmonizing Biographical Data 26214.6.1.1 Methodological Recommendations 26214.6.1.2 Procedural Recommendations 26314.6.1.3 Technical Recommendations 26314.6.2 Getting Started with the Cumulative HaSpaD Data Set 263Acknowledgments 264References 26415 Harmonization and Quality Assurance of Income and Wealth Data: The Case of LIS 269Jörg Neugschwender, Teresa Munzi, and Piotr R. Paradowski15.1 Introduction 26915.2 Applied Harmonization Methods 27115.3 Documentation and Quality Assurance 27515.3.1 Quality Assurance 275Selection of Source Datasets 276Harmonization 276Validation – “Green Light” Check 27615.3.2 Documentation 27815.4 Challenges to Harmonization 27815.5 Software Tools 28115.6 Conclusion 282References 28316 Ex-Post Harmonization of Time Use Data: Current Practices and Challenges in the Field 285Ewa Jarosz, Sarah Flood, and Margarita Vega-Rapun16.1 Introduction 28516.2 Applied Harmonization Methods 28916.2.1 Harmonizing the Matrix of the Diary 28916.2.2 Variable Harmonization 29116.2.3 Other Variables 29316.2.4 Other Types of Time Use Data 29416.3 Documentation and Quality Assurance 29416.3.1 Documentation 29416.3.2 Quality Checks 29616.4 Challenges to Harmonization 29716.5 Software Tools 30016.6 Recommendations 301References 302Part IV Further Issues: Dealing with Methodological Issues in Harmonized Survey Data 30517 Assessing and Improving the Comparability of Latent Construct Measurements in Ex-Post Harmonization 307Ranjit K. Singh and Markus Quandt17.1 Introduction 30717.2 Measurement and Reality 30717.3 Construct Match 30817.3.1 Consequences of a Mismatch 30917.3.2 Assessment 30917.3.2.1 Qualitative Research Methods 30917.3.2.2 Construct and Criterion Validity 30917.3.2.3 Techniques for Multi-Item Instruments 31017.3.2.4 Improving Construct Comparability 31117.4 Reliability Differences 31117.4.1 Consequences of Reliability Differences 31117.4.2 Assessment 31217.4.3 Improving Reliability Comparability 31217.5 Units of Measurement 31217.5.1 Consequences of Unit Differences 31317.5.2 Improving Unit Comparability 31317.5.3 Controlling for Instrument Characteristics 31417.5.4 Harmonizing Units Based on Repeated Measurements 31517.5.5 Harmonizing Units Based on Measurements Obtained from the Same Population 31517.6 Cross-Cultural Comparability 31617.6.1 Construct Match 31617.6.1.1 Translation and Cognitive Probing 31717.6.2 Reliability 31717.6.3 Units of Measurement 31817.6.3.1 Harmonizing Units of Localized Versions of the Same Instrument 31817.6.3.2 Harmonizing Units Across Cultures and Instruments 31817.6.4 Cross-Cultural Comparability of Multi-Item Instruments 31817.7 Discussion and Outlook 319References 32018 Comparability and Measurement Invariance 323Artur Pokropek18.1 Latent Variable Framework for Testing and Accounting for Measurement Non-Invariance 32418.2 Approaches to Empirical Assessment of Measurement Equivalence 32518.2.1 Classical Invariance Analysis (MG-CFA) 32618.2.2 Partial Invariance (MG-CFA) 32718.2.3 Approximate Invariance 32718.2.4 Approximate Partial Invariance (Alignment, BSEM Alignment, Partial BSEM) 32818.3 Beyond Multiple Indicators 32918.4 Conclusions 329References 33019 On the Creation, Documentation, and Sensible Use of Weights in the Context of Comparative Surveys 333Dominique Joye, Marlène Sapin, and Christof Wolf19.1 Introduction 33319.2 Design Weights 33519.2.1 What to do? 33619.3 Post-stratification Weights 33719.3.1 What Should be Done? 34019.4 Population Weights 34119.4.1 What Should be Done? 34219.5 Conclusion 342References 34420 On Using Harmonized Data in Statistical Analysis: Notes of Caution 347Claire Durand20.1 Introduction 34720.2 Challenges in the Combination of Data Sets 34720.2.1 A First Principle: A No Censorship Inclusive Approach 34820.2.2 A Second Principle: Using Multilevel Analysis and Introducing a Measurement Level 34920.2.3 A Third Principle: Assessing the Equivalence of Survey Projects 35120.3 Challenges in the Analysis of Combined Data Sets 35320.3.1 Dealing with Time 35420.3.2 Dealing with Missing Values 35820.3.2.1 Missing Values at the Respondent and Measurement Level 35820.3.2.2 Missing Values at the Survey Level 35920.3.3 Dealing with Weights 36120.4 Recommendations 362References 36321 On the Future of Survey Data Harmonization 367Kazimierz M. Slomczynski, Christof Wolf, Irina Tomescu-Dubrow, and J. Craig Jenkins21.1 What We Have Learned from Contributions on Survey Data Harmonization in this Volume 36821.2 New Opportunities and Challenges 37021.2.1 Reorientation of Survey Research in the Era of New Technology 37021.2.2 Advances in Technical Aspects of Data Management 37021.2.3 Harmonizing Survey Data with Other Types of Data 37121.3 Developing a New Methodology of Harmonizing Non-Survey Data 37221.3.1 Emerging Legal and Ethical Issues 37221.4 Globalization of Science and Harmonizing Scientific Practice 373References 373Index 377