Reliability and Validity of International Large-Scale Assessment

Dr Hans Wagemaker was the executive director of the International Association for the Evaluation of Educational Achievement for 17 years, responsible for the management of all IEA international research and assessment projects and activities. He helped develop IEA’s Progress in International Reading Literacy Study (PIRLS) and oversaw the development and expansion of IEA’s training and capacity building activities in low to middle income countries, and IEA’s educational consultancy services. Together with Educational Testing Services (ETS), he established the IEA Research Institute (IERI), where he continues to serve as a Board member. Dr Wagemaker was a Senior Manager Research and International with the Ministry of Education, New Zealand, and represented New Zealand’s interests in the APEC Education Forum, UNESCO’s commissions, and the OECD, CERI, and the Education Governing Board. He has consulted for the Inter American Development Bank and UNESCO and worked extensively with the World Bank to advance a common interest in the uses of assessment for improving educational systems in developing countries. Most recently Dr Wagemaker served as an advisor to the Minister of Education for the Sultanate of Oman. He is also a member of the Advisory Board for the Center for Education Statistics and Evaluation (CESE) for the government of New South Wales, Australia, the H Institute, Beirut, Lebanon, and continues in an advisory role with the IEA.Dr Wagemaker holds BA and MA degrees from the University of Otago, New Zealand, and a PhD from the University of Illinois, where he was awarded a University Fellowship and, in 2009, the College of Education’s Distinguished Alumni Award.

• Foreword; Dirk Hastedt.- 1 Introduction to Reliability and Validity of International Large-Scale Assessment; Hans Wagemaker.- 2 Study Design and Evolution, and the Imperatives of Reliability and Validity; Hans Wagemaker.- 3 Framework Development in International Large-Scale Assessment Studies; John Ainley and Wolfram Schulz.- 4 Assessment Content Development; Liz Twist and Julian Fraillon.- 5 Questionnaire Development in International Large-Scale Assessment Studies; Wolfram Schulz and Ralph Carstens.- 6 Translation: The Preparation of National Language Versions of Assessment Instruments; Paula Korsnakova, Steve Dept, and David Ebbs.- 7 Sampling, Weighting, and Variance Estimation; Sabine Meinck.- 8 Quality Control during Data Collection: Refining for Rigor; Lauren Musu, Sandra Dohr, and Andrea Netten.- 9 Post-collection Data Capture, Processing, and Scoring; Alena Becker.- 10 Technology and Assessment; Heiko Sibberns.- 11 Ensuring Validity in International Comparisons Using State-of-the-art Psychometric Methodologies; Matthias von Davier, Eugene Gonzalez, and Wolfram Schulz.- 12 Publishing and Dissemination; Seamus Hegarty and Sive Finlay.- 13 Consequential Validity: Data Access, Data Use, Analytical Support, and Training; Sabine Meinck, Eugene Gonzalez, and Hans Wagemaker.- 14 Using IEA Studies to Inform Policymaking and Program Development: The Case of Singapore; Hui Leng Ng, Chew Leng Poon, and Elizabeth Pang.- 15 Understanding the Policy Influence of International Large-Scale Assessments in Education; David Rutkowski, Greg Thompson, and Leslie Rutkowski.-