Online Panel Research

Mario Callegaro, Survey Research Scientist, Quantitative Marketing, Google Inc., UKReg Baker, President & Chief Operating Officer, Market Strategies International, USAPaul J. Lavrakas, Nielsen Media Research, Research Psychologist/Research Methodologist, USAJon A. Krosnick, Professor of Political Science, Communication, Psychology, Stanford University, USAJelke Bethlehem, Department of Quantitative Economics, University of Amsterdam, The NetherlandsAnja Göritz, University of Erlangen-Nuremberg, Department of Economics and Social Psychology, Germany

• Preface xvAcknowledgments xviiAbout the Editors xixAbout the Contributors xxiii1 Online panel research: History, concepts, applications and a look at the future 1Mario Callegaro, Reg Baker, Jelke Bethlehem, Anja S. Göritz, Jon A. Krosnick, and Paul J. Lavrakas1.1 Introduction 11.2 Internet penetration and online panels 21.3 Definitions and terminology 21.4 A brief history of online panels 41.5 Development and maintenance of online panels 61.6 Types of studies for which online panels are used 151.7 Industry standards, professional associations’ guidelines, and advisory groups 151.8 Data quality issues 171.9 Looking ahead to the future of online panels 172 A critical review of studies investigating the quality of data obtained with online panels based on probability and nonprobability samples 23Mario Callegaro, Ana Villar, David Yeager, and Jon A. Krosnick2.1 Introduction 232.2 Taxonomy of comparison studies 242.3 Accuracy metrics 272.4 Large-scale experiments on point estimates 282.5 Weighting adjustments 352.6 Predictive relationship studies 362.7 Experiment replicability studies 382.8 The special case of pre-election polls 422.9 Completion rates and accuracy 432.10 Multiple panel membership 432.11 Online panel studies when the offline population is less of a concern 462.12 Life of an online panel member 472.13 Summary and conclusion 48Part I COVERAGE 55Introduction to Part I 56Mario Callegaro and Jon A. Krosnick3 Assessing representativeness of a probability-based online panel in Germany 61Bella Struminskaya, Lars Kaczmirek, Ines Schaurer, and Wolfgang Bandilla3.1 Probability-based online panels 613.2 Description of the GESIS Online Panel Pilot 623.3 Assessing recruitment of the Online Panel Pilot 663.4 Assessing data quality: Comparison with external data 683.5 Results 743.6 Discussion and conclusion 804 Online panels and validity: Representativeness and attrition in the Finnish eOpinion panel 86Kimmo Grönlund and Kim Strandberg4.1 Introduction 864.2 Online panels: Overview of methodological considerations 874.3 Design and research questions 884.4 Data and methods 904.5 Findings 924.6 Conclusion 1005 The untold story of multi-mode (online and mail) consumer panels: From optimal recruitment to retention and attrition 104Allan L. McCutcheon, Kumar Rao, and Olena Kaminska5.1 Introduction 1045.2 Literature review 1075.3 Methods 1085.4 Results 1155.5 Discussion and conclusion 124Part II NONRESPONSE 127Introduction to Part II 128Jelke Bethlehem and Paul J. Lavrakas6 Nonresponse and attrition in a probability-based online panel for the general population 135Peter Lugtig, Marcel Das, and Annette Scherpenzeel6.1 Introduction 1356.2 Attrition in online panels versus offline panels 1376.3 The LISS panel 1396.4 Attrition modeling and results 1426.5 Comparison of attrition and nonresponse bias 1486.6 Discussion and conclusion 1507 Determinants of the starting rate and the completion rate in online panel studies 154Anja S. Göritz7.1 Introduction 1547.2 Dependent variables 1557.3 Independent variables 1567.4 Hypotheses 1567.5 Method 1637.6 Results 1647.7 Discussion and conclusion 1668 Motives for joining nonprobability online panels and their association with survey participation behavior 171Florian Keusch, Bernad Batinic, and Wolfgang Mayerhofer8.1 Introduction 1718.2 Motives for survey participation and panel enrollment 1738.3 Present study 1768.4 Results 1798.5 Conclusion 1859 Informing panel members about study results: Effects of traditional and innovative forms of feedback on participation 192Annette Scherpenzeel and Vera Toepoel9.1 Introduction 1929.2 Background 1939.3 Method 1969.4 Results 1999.5 Discussion and conclusion 207Part III MEASUREMENT ERROR 215Introduction to Part III 216Reg Baker and Mario Callegaro10 Professional respondents in nonprobability online panels 219D. Sunshine Hillygus, Natalie Jackson, and McKenzie Young10.1 Introduction 21910.2 Background 22010.3 Professional respondents and data quality 22110.4 Approaches to handling professional respondents 22310.5 Research hypotheses 22410.6 Data and methods 22510.7 Results 22610.8 Satisficing behavior 22910.9 Discussion 23211 The impact of speeding on data quality in nonprobability and freshly recruited probability-based online panels 238Robert Greszki, Marco Meyer, and Harald Schoen11.1 Introduction 23811.2 Theoretical framework 23911.3 Data and methodology 24211.4 Response time as indicator of data quality 24311.5 How to measure "speeding"? 24611.6 Does speeding matter? 25111.7 Conclusion 257Part IV WEIGHTING ADJUSTMENTS 263Introduction to Part IV 264Jelke Bethlehem and Mario Callegaro12 Improving web survey quality: Potentials and constraints of propensity score adjustments 273Stephanie Steinmetz, Annamaria Bianchi, Kea Tijdens, and Silvia Biffignandi12.1 Introduction 27312.2 Survey quality and sources of error in nonprobability web surveys 27412.3 Data, bias description, and PSA 27712.4 Results 28412.5 Potentials and constraints of PSA to improve nonprobability web survey quality: Conclusion 28613 Estimating the effects of nonresponses in online panels through imputation 299Weiyu Zhang13.1 Introduction 29913.2 Method 30213.3 Measurements 30313.4 Findings 30313.5 Discussion and conclusion 308Part V NONRESPONSE AND MEASUREMENT ERROR 311Introduction to Part V 312Anja S. Göritz and Jon A. Krosnick14 The relationship between nonresponse strategies and measurement error: Comparing online panel surveys to traditional surveys 313Neil Malhotra, Joanne M. Miller, and Justin Wedeking14.1 Introduction 31314.2 Previous research and theoretical overview 31414.3 Does interview mode moderate the relationship between nonresponse strategies and data quality? 31714.4 Data 31814.5 Measures 32014.6 Results 32414.7 Discussion and conclusion 33215 Nonresponse and measurement error in an online panel: Does additional effort to recruit reluctant respondents result in poorer quality data? 337Caroline Roberts, Nick Allum, and Patrick Sturgis15.1 Introduction 33715.2 Understanding the relation between nonresponse and measurement error 33815.3 Response propensity and measurement error in panel surveys 34115.4 The present study 34215.5 Data 34315.6 Analytical strategy 34415.7 Results 35015.8 Discussion and conclusion 357Part VI SPECIAL DOMAINS 363Introduction to Part VI 364Reg Baker and Anja S. Göritz16 An empirical test of the impact of smartphones on panel-based online data collection 367Frank Drewes16.1 Introduction 36716.2 Method 36916.3 Results 37116.4 Discussion and conclusion 38517 Internet and mobile ratings panels 387Philip M. Napoli, Paul J. Lavrakas, and Mario Callegaro17.1 Introduction 38717.2 History and development of Internet ratings panels 38817.3 Recruitment and panel cooperation 39017.4 Compliance and panel attrition 39417.5 Measurement issues 39617.6 Long tail and panel size 39817.7 Accuracy and validation studies 40017.8 Statistical adjustment and modeling 40117.9 Representative research 40217.10 The future of Internet audience measurement 403Part VII OPERATIONAL ISSUES IN ONLINE PANELS 409Introduction to Part VII 410Paul J. Lavrakas and Anja S. Göritz18 Online panel software 413Tim Macer18.1 Introduction 41318.2 What does online panel software do? 41418.3 Survey of software providers 41518.4 A typology of panel research software 41618.5 Support for the different panel software typologies 41718.6 The panel database 41818.7 Panel recruitment and profile data 42118.8 Panel administration 42318.9 Member portal 42518.10 Sample administration 42818.11 Data capture, data linkage and interoperability 43018.12 Diagnostics and active panel management 43318.13 Conclusion and further work 43619 Validating respondents’ identity in online samples: The impact of efforts to eliminate fraudulent respondents 441Reg Baker, Chuck Miller, Dinaz Kachhi, Keith Lange, Lisa Wilding-Brown, and Jacob Tucker19.1 Introduction 44119.2 The 2011 study 44319.3 The 2012 study 44419.4 Results 44619.5 Discussion 44919.6 Conclusion 450References 451Appendix 19.A 452Index 457