Uneducated Guesses

Howard Wainer is distinguished research scientist at the National Board of Medical Examiners and adjunct professor of statistics at the Wharton School of the University of Pennsylvania. For twenty-one years, he was principal research scientist at Educational Testing Service. His many books include "Picturing the Uncertain World: How to Understand, Communicate, and Control Uncertainty through Graphical Display" and "Graphic Discovery: A Trout in the Milk and Other Visual Adventures" (both Princeton).

• Preface xi Introduction 1 Chapter 1: O n the Value of Entrance Exams: What Happens When the SAT Is Made Optional? 8 Chapter 2: O n Substituting Achievement Tests for Aptitude Tests in College Admissions 20 Chapter 3: O n Rigid Decision Rules for Scholarships 29 Chapter 4: The Aptitude-Achievement Connection: Using an Aptitude Test to Aid in Allocating Educational Resources 32 Chapter 5: C omparing the Incomparable: On the Importance of Big Assumptions and Scant Evidence 57 Chapter 6: O n Examinee Choice in Educational Testing 73 Chapter 7: What If Choice Is Part of the Test? 103 Chapter 8: A Little Ignorance Is a Dangerous Thing: How Statistics Rescued a Damsel in Distress 110 Chapter 9: A ssessing Teachers from Student Scores: On the Practicality of Value-Added Models 120 Chapter 10: S hopping for Colleges When What We Know Ain't 139 Chapter 11: O f CAT s and Claims: The First Step toward Wisdom 147 Epilogue 156 References 159 Index 165

Winner of the 2014 AERA Division D Significant Contribution to Educational Measurement and Research Methodology Award, American Educational Research Association "[T]hought-provoking... He questions the anecdotal and statistical evidence that underpins many of today's education policies and reform efforts."--Library Journal "An absolutely absorbing book. Feels like a must for politicians, reformers, educators--math educators in particular."--Cut the Knot Insights "Renowned statistician and research scientist Howard Wainer applies the tools of his trade to answer a question that affects every American: What is wrong with our education system? ... Wainer pokes holes in almost every aspect of conventional education policy--college rankings, admissions, aptitude tests--including a scathing critique of No Child Left Behind."--Bruce Walsh, Metro "[Wainer's] overall message rings clear and true for much more than assessment: Policy that is formed without full analysis of the breadth of data available on a topic is policy that will fail."--Laurent Rigal, Education Gadfly "Tired of yelling at the TV when he saw news accounts of policy changes based on flawed evidence, Wainer uses his book to present evidence to help assess 11 such trends, including the entrance-exam-optional policies in many colleges and teacher evaluations based on student performance... Wainer applies more than statistical evidence to education policy; he also brings common sense to bear."--Maureen Downey, Atlanta Journal-Constitution "With its timely reminder that high stakes decisions often rely on anecdotes, laden with emotion, and that 'the plural of anecdote is not data,' Uneducated Guesses ought to be read by anyone who is concerned about the weaknesses (and wrong-headed assumptions) in current educational policies."--Glenn C. Altschuler, Tulsa World "I would recommend this book to anyone with an interest in testing, especially for college admissions or advanced placement... Wainer is a gifted writer with a notable talent for analyzing and presenting data."--Bill Satzer, MAA Reviews "The book provides a model for the development of rational public education policies, something that America needs desperately."--Robert A. Bligh, Education Review "Educators and education policymakers interested in helping students realize their potential will benefit from reading Wainer's book because the implications reach beyond postsecondary school instruction. Teachers and administrators at all levels can follow the logic of Wainer's ideas as they seek to use evidenced-based pedagogical strategies in their classrooms."--Denise G. Brassell, Mathematics Teacher "In sum, this book is a wonderful compilation of concrete examples from educational testing that amply illustrate the importance of evidence-based policy-making. I recommend it as an interesting, entertaining, and most worthwhile read."--TCR Books