Advancing Natural Language Processing in Educational Assessment

Victoria Yaneva is Senior NLP Scientist at the National Board of Medical Examiners, USA.Matthias von Davier is Monan Professor of Education in the Lynch School of Education and Executive Director of TIMSS & PIRLS International Study Center at Boston College, USA.

• Preface by Victoria Yaneva and Matthias von DavierSection I: Automated ScoringChapter 1: The Role of Robust Software in Automated Scoring by Nitin Madnani, Aoife Cahill, and Anastassia LoukinaChapter 2: Psychometric Considerations when Using Deep Learning for Automated Scoring by Susan Lottridge, Chris Ormerod, and Amir JafariChapter 3: Speech Analysis in Assessment by Jared C. Bernstein and Jian ChengChapter 4: Assessment of Clinical Skills: A Case Study in Constructing an NLP-Based Scoring System for Patient Notes by Polina Harik, Janet Mee, Christopher Runyon, and Brian E. ClauserSection II: Item DevelopmentChapter 5: Automatic Generation of Multiple-Choice Test Items from Paragraphs Using Deep Neural Networks by Ruslan Mitkov, Le An Ha, Halyna Maslak, Tharindu Ranasinghe, and Vilelmini SosoniChapter 6: Training Optimus Prime, M.D.: A Case Study of Automated Item Generation using Artificial Intelligence – From Fine-Tuned GPT2 to GPT3 and Beyond by Matthias von DavierChapter 7: Computational Psychometrics for Digital-first Assessments: A Blend of ML and Psychometrics for Item Generation and Scoring by Geoff LaFlair, Kevin Yancey, Burr Settles, Alina A von DavierSection III: Validity and FairnessChapter 8: Validity, Fairness, and Technology-based Assessment by Suzanne LaneChapter 9: Evaluating Fairness of Automated Scoring in Educational Measurement by Matthew S. Johnson and Daniel F. McCaffreySection IV: Emerging TechnologiesChapter 10: Extracting Linguistic Signal from Item Text and Its Application to Modeling Item Characteristics by Victoria Yaneva, Peter Baldwin, Le An Ha, and Christopher RunyonChapter 11: Stealth Literacy Assessment: Leveraging Games and NLP in iSTART by Ying Fang, Laura K. Allen, Rod D. Roscoe, and Danielle S. McNamaraChapter 12: Measuring Scientific Understanding Across International Samples: The Promise of Machine Translation and NLP-based Machine Learning Technologies by Minsu Ha and Ross H. NehmChapter 13: Making Sense of College Students’ Writing Achievement and Retention with Automated Writing Evaluation by Jill Burstein, Daniel McCaffrey, Steven Holtzman & Beata Beigman KlebanovContributor Biographies