Psychological Testing

Thomas P. Hogan, Ph.D., is Distinguished University Professor Emeritus in the Psychology Department, University of Scranton. He is the author/co-author of over 200 published papers, chapters, and presentations, as well as four books, on psychological testing, measurement, and methodology; and is author/co-author of several nationally used standardized tests. He has frequently served as consultant to national organizations on assessment issues. He holds a Ph.D. in psychology with specialization in psychometrics from Fordham University.

• Preface iiiPart 11 The World of Psychological Testing, 3Introduction, 3Major Categories of Tests, 4Some Additional Ways to Categorize Tests, 6Uses and Users of Tests, 8Major Issues: Assumptions and Questions, 10Basic Assumptions, 10Fundamental Questions, 12The Differential Perspective, 13The Historical Perspective, 13Remote Background: Up to 1840, 14Setting the Stage: 1840–1880, 15The Roots: 1880–1915, 17The Flowering: 1915–1940, 19Consolidation: 1940–1965, 21Just Yesterday: 1965–2000, 22And Now: 2000–Present, 23Major Forces, 24The Scientific Impulse, 24Concern for the Individual, 24Practical Applications, 25Statistical Methodology, 25The Rise of Clinical Psychology, 25Computers, 25By Way of Definition, 28Summary, 29Key Terms, 30Exercises, 302 Sources of Information About Tests, 32Two Common Problems Requiring Information About Tests, 32A Test’s Introductory Kit, 33Comprehensive Lists of Tests, 34Test Collection at ETS, 34PsycTESTS, 35Health and Psychosocial Instruments (HaPI), 35Tests in Print, 36Systematic Reviews, 37Buros Reviews Available Electronically, 37Some Other Sources of Reviews, 38Special-Purpose Collections, 38Books About Single Tests, 39Textbooks on Testing, 40Journals, 40Publishers’ Catalogs and Personnel, 42Other Users, 42Strengths and Shortcomings of the Sources, 43Summary, 44Key Terms, 44Exercises, 443 Test Norms, 47Purpose of Norms, 47Review of Statistics: Part 1, 48Variables, 48Types of Scales, 49Organization of Raw Data, 50Central Tendency, 51Variability, 52z-Scores, 53Shapes of Distributions, 54The Raw Score, 55The Special Case of Theta (θ), 56Types of Norms, 57Percentile Ranks and Percentiles, 59Standard Scores, 63Developmental Norms, 69Examples of Norm Tables, 72Interpretive Reports and Norms, 72Innovative Ways to Interpret Test Performance, 74What to Read?, 74Ready or Not?, 75Depressed or Not?, 75What Job for You?, 75Conclusion, 75Norm Groups, 76National Norms, 76International Norms, 76Convenience Norm Groups, 76User Norms, 77Subgroup Norms, 77Local Norms, 77Institutional Norms, 78Criterion-Referenced Interpretation, 79The Standardization Group: Determining Its Usefulness, 80Summary, 83Key Terms, 84Exercises, 844 Reliability, 86Introduction, 86Four Important Distinctions, 87Review of Statistics: Part 2—Correlation and Prediction, 88Bivariate Distribution and Correlation Coefficients, 88Regression Line, 90Factors Affecting Correlation Coefficients, 92Major Sources of Unreliability, 96Test Scoring, 96Test Content, 98Test Administration Conditions, 98Personal Conditions, 99Conceptual Framework: True Score Theory, 99Methods of Determining Reliability, 101Test–Retest Reliability, 101Inter-Scorer Reliability, 102Alternate Form Reliability, 103Internal Consistency Reliability, 104Split-Half Reliability, 104Kuder–Richardson Formulas, 105Coefficient Alpha, 106Three Important Conclusions, 108The Standard Error of Measurement, 108Confidence Bands, 109Appropriate Units for SEM, 109Standard Errors: Three Types, 110Some Special Issues in Reliability, 111Reliability in Item Response Theory, 113Generalizability Theory, 114Factors Affecting Reliability Coefficients, 115How High Should Reliability Be?, 115Summary, 116Key Terms, 117Exercises, 1175 Validity, 119Introduction, 119Refining the Definition of Validity, 120Construct Underrepresentation and Construct-Irrelevant Variance, 121The Basic Issue, 122The Traditional and Newer Classifications of Types of Validity Evidence, 123The Issue of Face Validity, 124Content Validity, 124Application to Achievement Tests, 124Instructional Validity, 126Application to Employment Tests, 127Content Validity in Other Areas, 128Problems with Content Validity, 128Criterion-Related Validity, 129External, Realistic Criterion, 130Contrasted Groups, 132Correlations with Other Tests, 133Special Considerations for Interpreting Criterion-Related Validity, 134The Reliability–Validity Relationship, 135Combining Information from Different Tests, 138Decision Theory: Basic Concepts and Terms, 141Hits, False Positives, and False Negatives, 142Base Rate, 143Sensitivity and Specificity, 144Construct Validity, 145Internal Structure, 146Factor Analysis, 146Response Processes, 148Effect of Experimental Variables, 148Developmental Changes, 149Consequential Validity, 149Test Bias as Part of Validity, 150The Practical Concerns, 151Integrating the Evidence, 151In the Final Analysis: A Relative Standard, 152Summary, 152Key Terms, 153Exercises, 1536 Test Development and Item Analysis, 155Introduction, 155Defining the Test’s Purpose, 156Preliminary Design Issues, 157Origin of New Tests, 158Item Preparation, 159Types of Test Items, 160Selected-Response Items, 160Scoring Selected-Response Items, 162Constructed-Response Items, 162Scoring Constructed-Response Items, 163The Pros and Cons of Selected-Response versus Constructed-Response Items, 166Suggestions for Writing Selected-Response Items, 167Suggestions for Writing Constructed-Response Items, 167Some Practical Considerations in Writing Items, 168Technology-based Innovations in Item Structure, 169Item Analysis, 169Item Tryout, 170Item Statistics, 170Item Difficulty, 171Item Discrimination, 171Examples of Item Statistics, 172Item Statistics in Item Response Theory, 174Factor Analysis as an Item Analysis Technique, 177Item Selection, 178Computer-Adaptive Testing, 181Standardization and Ancillary Research Programs, 183Preparation of Final Materials and Publication, 184Summary, 185Key Terms, 185Exercises, 1857 Fairness and Bias, 187Fairness: Gaining Perspective, 187Methods of Studying Test Fairness, 190Panel Review, 191Differential Item Functioning, 192Differential Prediction, 194Measurement Invariance, 196Accommodations and Modifications, 197Research on Accommodations, 198Some Tentative Conclusions about Test Fairness, 199Summary, 200Key Terms, 201Exercises, 201Part 28 Cognitive Abilities: Individual Tests, 205Some Cases, 205Introduction to Cognitive Ability Tests, 206Some Terminology, 206Real-World Correlates of Cognitive Abilities, 207Structure of Cognitive Abilities, 208Uses and Characteristics of Individual Cognitive Ability Tests, 211Typical Items in an Individual Intelligence Test, 213The Wechsler Scales: An Overview, 215Historical Introduction, 215Weschsler’s Concept of Intelligence, 216Wechsler Adult Intelligence Scale—Fourth Edition, 216Wechsler Intelligence Scale for Children—Fifth Edition, 223WISC versus WAIS, 223Psychometric Characteristics of WISC-V, 224The Stanford-Binet, 225Transition to New Structure: SB4 and SB5, 225Psychometric Characteristics of SB5, 226Brief Individually Administered Tests of Mental Ability, 227Peabody Picture Vocabulary Test, 227Two Other Entries, 231A Test of a Specific Cognitive Ability: The Wechsler Memory Scale, 231Intellectual Disability, 235Changing Terminology, 236The Concept of Adaptive Behavior, 236Definition of Intellectual Disability, 237Vineland Adaptive Behavior Scales, 238Other Applications of Adaptive Behavior, 241Infant and Early Childhood Tests, 241Other Areas for Cognitive Ability Tests, 241Trends in Individually Administered Cognitive Ability Tests, 242Summary, 244Key Terms, 244Exercises, 2449 Cognitive Abilities: Group Tests, 246Some Cases, 246Uses of Group-Administered Cognitive Ability Tests, 247Common Characteristics of Group Cognitive Ability Tests, 248Cognitive Ability Tests in School Testing Programs, 250Otis-Lennon School Ability Test, 250College Admissions Tests, 257The SAT, 257The ACT, 259Graduate and Professional School Selection, 264Graduate Record Examinations: General Test, 265Military and Business Selection Tests, 269Armed Services Vocational Aptitude Battery, 270Wonderlic Personnel Test, 272Culture-Fair Tests of Cognitive Ability, 273Intelligence Tests for Microcultures, 275Generalizations About Group Cognitive Ability Tests, 276Summary, 277Key Terms, 278Exercises, 27810 Neuropsychological Assessment, 279Case Examples, 279Focus on the Brain: The Road to Clinical Neuropsychology, 280Two Main Approaches to Neuropsychological Assessment, 284Fixed Battery Approach, 284Luria-Nebraska Neuropsychological Battery, 284Halstead-Reitan Neuropsychological Test Battery, 285Flexible Battery Approach, 287Mental Status, 288Intelligence, 289Achievement, 289Attention/Concentration, 289Language, 290Visuospatial/Perceptual, 291Memory, 292Motor Functioning, 294Executive Functions, 295Personality/Psychological State, 296Supplementary Information, 298Medical History, 299Psychiatric History, 299Psychosocial History, 299School Records, 300Collateral Information, 300Behavioral Observations, 300Case Examples Revisited, 301Summary, 305Key Terms, 305Exercises, 30611 Achievement Tests, 307Introduction, 307The Ability-Achievement Continuum, 308The Psychologist’s Interface with Achievement Tests, 308A Broad Classification of Achievement Tests, 309A Typical School Testing Program, 310The Accountability Movement and Standards-Based Education, 310Trends in Achievement Testing in the Schools, 311Achievement Batteries, 312Stanford Achievement Test, 312Typical Uses and Special Features, 314Achievement Batteries at the College Level, 315Single-Area Achievement Tests, 316Examples, 316Typical Uses and Special Features, 318Licensing and Certification Tests, 318Examples, 319Typical Uses and Special Features, 319A Primer on Establishing Cutoff Scores, 320State, National, and International Achievement Tests, 321State Testing Programs, 321A National Testing Program: NAEP, 322International Testing Programs: TIMSS, PIRLS, PISA, and PIAAC, 322Special Features, 323Individually Administered Achievement Tests, 323Examples, 324Typical Uses and Special Features, 326Curriculum Based Measures, 327General Characteristics, 327Examples, 327Interpretation, 328Some Nagging Questions about Achievement Tests, 329Summary, 330Key Terms, 331Exercises, 33112 Objective Personality Tests, 333Introduction, 333Uses of Objective Personality Tests, 334A Functional Classification of Objective Personality Tests, 335Comprehensive Inventories: Common Characteristics, 336Specific Domain Tests: Common Characteristics, 338The Special Problems of Response Sets and Faking, 339Strategies for Dealing with Response Sets and Faking, 340Major Approaches to Personality Test Development, 343Content Method, 343Criterion-Keying Approach, 344Factor Analysis, 346Theory-Driven Approach, 346Combined Approaches, 347Examples of Comprehensive Inventories, 347The Edwards Personal Preference Schedule (EPPS): An Example of a Theory-Based Test, 348The NEO Personality Inventory-3: An Example of a Factor-Analytic Test, 349IPIP: Build Your Own Personality Inventory, 352Specific Domain Tests, 353Piers-Harris Children’s Self-Concept Scale, 353Measures within Positive Psychology, 355An Example: Subjective Well-Being, 356Trends in the Development and Use of Objective Personality Tests, 357Summary, 358Key Terms, 359Exercises, 35913 Clinical Instruments and Methods, 360Introduction, 360The Clinical Interview as Assessment Technique, 361Unstructured, Semistructured, and Structured Interviews, 361The DSM and ICD, 362Categorical versus Dimensional Approaches, 362Structured Clinical Interview for DSM-IV Axis I Disorders, 363The Employment Interview: A Sidebar, 364Examples of Comprehensive Self-Report Inventories, 365The Minnesota Multiphasic Personality Inventory (MMPI), 365MMPI-2 RF (Restructured Form), 372The Millon Clinical Multiaxial Inventory (MCMI) and the Millon Family, 373Personality Assessment Inventory (PAI), 376Symptom Checklist-90-R, 378Examples of Specific Domain Tests, 380The Beck Depression Inventory (BDI), 380The Eating Disorder Inventory (EDI), 381State-Trait Anxiety Inventory, 384Behavior Rating Scales, 385Multiscore Systems, 386Single-Area Scales, 387Behavioral Assessment, 387Direct or Naturalistic Observation, 388Analogue Behavioral Observation, 388Behavioral Interviewing, 389Self-Monitoring and Self-Report, 389Cognitive-Behavioral Assessment, 390Physiological Measures, 390Concluding Comments on Behavioral Assessment Methods, 390Trends in the Development and Use of Clinical Instruments, 391Summary, 392Key Terms, 392Exercises, 39314 Projective Techniques, 394General Characteristics of Projective Techniques and the Projective Hypothesis, 394Uses of Projective Techniques, 395Indicators for the Use of Projectives, 397Administration and Scoring of Projective Techniques: A Forewarning,397The Rorschach Inkblot Test, 398The Materials, 399Administration and Scoring, 400The Coding System, 402Sequence of Scores and Structural Summary, 404Evaluation of the Rorschach, 404Rorschach Performance Assessment System (R-PAS), 405Thematic Apperception Test (TAT), 406Rotter Incomplete Sentences Blank (RISB), 408Human Figure Drawings, 411The Future of Projective Techniques, 413Training of Psychologists, 413Emergence of Managed Care, 413Objective Scoring, Norm-Referenced Interpretation, and PsychometricQuality, 414Summary, 415Key Terms, 415Exercises, 41515 Interests and Attitudes, 417Introduction, 417Orientation to Career Interest Testing, 418Strong and Kuder, 418Traditional Approaches, 418Uses of Career Interest Tests, 419A Forewarning on Names, 420Holland Themes and the RIASEC Codes, 421Strong Interest Inventory, 422Kuder Career Interests Assessments, 427Self-Directed Search (SDS), 428Some Generalizations about Career Interest Measures, 431Attitude Measures, 432Likert Scales, 433Thurstone Scales, 435Guttman Scales, 436Public Opinion Polls and Consumer Research, 437Summary, 437Key Terms, 438Exercises, 43816 Ethical and Legal Issues, 439Ethics versus Law, 439Ethical Issues, 440Background on Professional Ethics, 440Sources of Ethical Principles for Testing, 441Generalizations about Ethical Use of Tests, 444Competence, 444Informed Consent, 444Knowledge of Results, 445Confidentiality, 445Test Security, 445Test Development and Publication, 445Automated Scoring/Interpretation Systems, 446Unqualified Persons, 446Test User Qualifications, 446Legal Issues, 447Areas of Application: An Overview, 448Definition of Laws, 448Laws Related to Testing, 449The Fourteenth Amendment, 450The Civil Rights Acts of 1964 and 1991, 451Rehabilitation Act of 1973 and Americans withDisabilities Act of 1990, 451The Handicapped/Disabled in Education: P.L. 94–142 and IDEA, 452FERPA and HIPAA, 453EEOC Guidelines, 454ESEA, NCLB, and ESSA, 456Illustrative Court Cases, 456Griggs v. Duke Power, 457Debra P. v. Turlington and GI Forum v. TEA, 458Larry P. v. Riles, PASE v. Hannon, and Crawford v. Honig, 459Karraker v. Rent-A-Center, 460Atkins v. Virginia and Hall v. Florida, 460New Haven Firefighters Case, 461Forensic Application of Tests, 461Two Legal Terms, 462Three Areas of Special Concern, 462And Beyond, 463Some Generalizations about the Interface of Testing and the Law, 463Summary, 464Key Terms, 465Exercises, 465Appendix A. Test Reviewing and Selection 467Appendix B. How to Build a (Simple) Test 473Appendix C. Contact Information for Major Test Publishers 479Appendix D. Sample Data Sets 480Appendix E. Answers to Selected Exercises 481Glossary 485References 501Name Index 523Subject Index 529