ABCs of Evaluation

John Boulmetis is a professor in the School of Education (SOE), director of graduate studies for the SOE, co-director of the URI/RIC PhD Program, and coordinator of the graduate program in Adult Education at the University of Rhode Island.Phyllis Dutwin is president of Dutwin Associates, a training and development firm, and former vice president of Reading and Educational Services of New Jersey, and is the author of more than a dozen books used by adults individually and in adult education classrooms.

• Preface viiThe Authors xv1 What is Evaluation? 1Common Denominators 3Two Definitions of Evaluation 4Evaluating Efficiency, Effectiveness, and Impact 5Evaluating Alternatives 8Identifying Areas to Improve 8Other Levels 10Formal Reasons to Evaluate 12Practical Application 15Formative and Summative Evaluations 16Putting It All Together 16Key Words and Concepts 24Questions and Exercises 24Further Reading 272 Why Evaluate? 29Grants and Grantsmanship 31Benefits 33Limitations 35Considerations 36Putting It All Together 40Key Words and Concepts 40Questions and Exercise 43Further Reading 463 Decision Making: Whom to Involve, How, and Why? 47Program and Project Cycles 49Evaluation and Monitoring 52Program Planning Cycle and Evaluation 57Putting It All Together 61Key Words and Concepts 65Questions and Exercises 65Further Reading 734 Starting Point: The Evaluator’s Program Description 75What an Evaluator’s Program Description Can Do 77Developing an Evaluator’s Program Description 78Putting It All Together 83Key Words and Concepts 84Questions and Exercises 84Further Reading 965 Choosing an Evaluation Model 97More Than One Way to Evaluate 98Four Evaluation Models 101Other Models 108Choosing the Right Model 110Evaluation Design Format 111Other Considerations 114Putting It All Together 114Key Words and Concepts 115Questions and Exercise 115Further Reading 1176 Data Sources 119Starting Point 120Qualitative and Quantitative Data Sources 124Four Levels of Data 125Nature of Data Collection 127Existing Data 128Newly Collected Data 130Putting It All Together 141Key Words and Concepts 141Questions and Exercise 141Further Reading 1447 Data Analysis 147Data Analysis 148Data 150Terms 151Levels of Data 157Putting It All Together 164Key Words and Concepts 164Questions and Exercise 165Further Reading 167Software for Personal Computers 1688 Is It Evaluation or Is It Research? 169Determining Evaluation or Research from Intent 170Determining Evaluation or Research from Design 172Qualitative and Quantitative Characteristics 174Sampling 175Sampling and Surveys 180Putting It All Together 181Key Words and Concepts 183Questions and Exercise 183Further Reading 1849 Writing the Evaluation Report 185Communicating Your Findings 187Evaluation Reports 188The Report Outline 192Putting It All Together 201Key Words and Concepts 202Questions and Exercise 202Further Reading 20310 Evaluation as a Business 205Is There a Market for Program Evaluation? 206How Do You Find the Business? 211Proactive Resources 213Reactive Resources 216Emergency Resources 216How Do You Negotiate a Contract? 217Keeping the Business Alive 222Putting It All Together 224Key Words and Concepts 225Further Reading 225A: More on Data Analysis 227Analyses for Each Level 227Significance Testing 231Statistical and Practical Significance 235B: Sample Evaluation Report 237C: Sample Evaluation Contract 271References 277Index 281