Statistics, Testing, and Defense Acquisition

For every weapons system being developed, the U.S. Department of Defense (DOD) must make a critical decision: Should the system go forward to full-scale production? The answer to that question may involve not only tens of billions of dollars but also the nation's security and military capabilities. In the milestone process used by DOD to answer the basic acquisition question, one component near the end of the process is operational testing, to determine if a system meets the requirements for effectiveness and suitability in realistic battlefield settings. Problems discovered at this stage can cause significant production delays and can necessitate costly system redesign.This book examines the milestone process, as well as the DOD's entire approach to testing and evaluating defense systems. It brings to the topic of defense acquisition the application of scientific statistical principles and practices.Table of ContentsFront MatterExecutive Summary1 Introduction2 Operational Testing and System Acquisition3 A New Paradigm for Testing and Evaluation in Defense Acquisition4 Upgrading Statistical Methods for Testing and Evaluation5 Improving Operational Test Planning and Design6 Analyzing and Reporting Test Results7 Assessing Operational Suitability8 Testing Software-Intensive Systems9 Using Modeling and Simulation in Test Design and Evaluation10 Increasing Access to Statistical Expertise for Operational TestingAppendix A Case Studies and System DescriptionsAppendix B Abstracts of Background PapersAppendix C A Potential Taxonomy for Operational TestsAppendix D Elements of ISO 9000Appendix E Glossary and AcronymsReferencesBiographical Sketches of Panel Members and Staff