Assessing the Reliability of Complex Models

Advances in computing hardware and algorithms have dramatically improved the ability to simulate complex processes computationally. Today's simulation capabilities offer the prospect of addressing questions that in the past could be addressed only by resource-intensive experimentation, if at all. Assessing the Reliability of Complex Models recognizes the ubiquity of uncertainty in computational estimates of reality and the necessity for its quantification. As computational science and engineering have matured, the process of quantifying or bounding uncertainties in a computational estimate of a physical quality of interest has evolved into a small set of interdependent tasks: verification, validation, and uncertainty of quantification (VVUQ). In recognition of the increasing importance of computational simulation and the increasing need to assess uncertainties in computational results, the National Research Council was asked to study the mathematical foundations of VVUQ and to recommend steps that will ultimately lead to improved processes. Assessing the Reliability of Complex Models discusses changes in education of professionals and dissemination of information that should enhance the ability of future VVUQ practitioners to improve and properly apply VVUQ methodologies to difficult problems, enhance the ability of VVUQ customers to understand VVUQ results and use them to make informed decisions, and enhance the ability of all VVUQ stakeholders to communicate with each other. This report is an essential resource for all decision and policy makers in the field, students, stakeholders, UQ experts, and VVUQ educators and practitioners. Table of Contents Front Matter Summary 1 Introduction 2 Sources of Uncertainty and Error 3 Verification 4 Emulation, Reduced-Order Modeling, and Forward Propagation 5 Model Validation and Prediction 6 Making Decisions 7 Next Steps in Practice, Research, and Education for Verification, Validation, and Uncertainty Quantification Appendixes Appendix A: Glossary Appendix B: Agendas of Committee Meetings Appendix C: Committee Biographies Appendix D: Acronyms