Research Handbook on Program Evaluation

Edited by Kathryn E. Newcomer, Professor of Public Policy and Public Administration, Trachtenberg School of Public Policy & Public Administration, The George Washington University and Steven W. Mumford, Assistant Professor of Political Science, Department of Political Science, University of New Orleans, US

• ContentsIntroduction: key challenges for evaluation theory and practice 1Kathryn E. Newcomer and Steven W. MumfordPART I THEORETICAL ISSUES IN PROGRAM EVALUATION1 Evaluation use and influence 14Melvin M. Mark2 Evaluative thinking 35Thomas Archibald3 The evaluation society and its dangers 50Peter Dahler-Larsen4 Values and valuing 66Thomas A. Schwandt5 Multicultural validity: importance and grounded application 84Cinthia Josette Arévalo and Louise Moana WerePART II THEORETICAL APPROACHES TO PROGRAM EVALUATION6 Cultural responsiveness in evaluation 98Ayesha S. Boyce, Jori N. Hall and Cherie M. Avent7 Framing participatory evaluation: a twenty-five-year retrospective 116J. Bradley Cousins and Elizabeth Whitmore8 Using principles to guide evaluation theory and practice 137Michael Quinn Patton and Charmagne E. Campbell-Patton9 It’s about more than equity: Equitable Evaluation Framework® 161Jara Dean-Coffey10 Feminist principles evaluation 174Donna Rae Podems11 Cross-cultural evaluation in Indigenous communities: a journey intoand out of colonized spaces 192Jill Anne Chouinard and Susanne ThiessenPART III ONGOING TRENDS IN PROGRAM EVALUATION12 Engagement in evaluation 211Esther C. Nolton and Laura P. Forsythe13 Building evaluation capacity 240Tessie Catsambas and Ghazia Aslam14 Program theory 260Brad Astbury15 Mixing methods in program evaluation 280Sarah Mason and Steven W. Mumford16 Exploring causal pathways amid complexity 304Jewlya Lynn and Marina Apgar17 Evaluating advocacy initiatives 326Sarah E. Stachowiak and Elizabeth B. Taylor-Schiro – Biidabinikwe18 Evaluating complex development programs: integrating complexitythinking and systems analysis 348Michael Bamberger and Aaron E. Zazueta19 Evidence-informed policy and management? Disparate communities ofpractice providing information to government 374Clinton T. Brass and Kathryn E. Newcomer20 Experimental approaches in evaluation 397Laura R. Peck21 Evidence-building and evaluation in government 424Nicholas Hart and Kathryn E. NewcomerPART IV EMERGING PRACTICES IN PROGRAM EVALUATION22 Strategic evaluation agendas: a case study from the Netherlands 443Nynke de Witte, Carolien Klein Haarhuis, Rien Rouw and Odette van de Riet23 Using multilevel models to address intersectionality in program evaluation 460Eiko Hiraoka Strader24 Emerging trust-based evaluation approaches in philanthropy 475Julia Coffman and Chera Reid25 Evaluating social enterprises: research and practice 489Saurabh A. Lall and Kaushal Sapkota26 Real-time evaluation: past and potential 508Scott G. Chaplowe27 Utilizing narratives to examine equity 530Justine Augeri28 Enhancing evaluation with Q-methodology 549Chad Oatley, Kevin Harris and Steven Henderson29 Process tracing methods in evaluation 570Estelle Raimondo and Derek Beach30 Synergies between program evaluation and lean management 587Yvonne M. Watson31 Serious games for evaluation 608Karol Olejniczak and Igor Widawski32 Crowdsourcing in evaluation 631Tarek Azzam33 Bringing a futures mindset to evaluation 649Jewlya Lynn, Katri Vataja and Rose Thompson Coon34 Using big data to strengthen evaluation 667Michael BambergerConclusion: key trends shaping the field 687Steven W. Mumford and Kathryn E. NewcomerIndex 692

‘This book deserves a place on the bookshelf of everyone engaged in evaluation- evaluators, evaluation managers, evaluation educators and evaluation researchers. It provides comprehensive and up to date coverage of key concepts and practices in evaluation, such as evaluation use, evaluative thinking, values in evaluation, using theories of change, mixed methods, experimental designs, engagement and participation. It also provides a useful introduction to applications, methods and concepts that might be less familiar such as principles-based evaluation, equitable evaluation, real-time evaluation, futures thinking and big data. The distinguishing feature of the book is the clear and thorough coverage of the issues, providing an overview of its key concepts, development over time and variations, as well as the personal insights from the experienced and informed authors of each chapter. Reading through each chapter feels like being in a masterclass on the topic.’