DR. RAJESH JUGULUM, Ph.D., is a Data Quality and Analytics Professional. Rajesh held executive positions in these fields at Citi Group and Bank of America. Before joining financial industry, Rajesh was with MIT where he was involved in research and teaching. Currently, he teaches at Northeastern University in Boston. His honors include 2002 American Society for Quality’s Feigenbaum medal and 2006 International Technology Institute’s Rockwell medal.

• Foreword xiii Prelude xvPreface xviiAcknowledgments xix1 The Importance of Data Quality 11.0 Introduction 11.1 Understanding the Implications of Data Quality 11.2 The Data Management Function 41.3 The Solution Strategy 61.4 Guide to This Book 6Section I Building a Data Quality Program 2 The Data Quality Operating Model 132.0 Introduction 132.1 Data Quality Foundational Capabilities 132.1.1 Program Strategy and Governance 142.1.2 Skilled Data Quality Resources 142.1.3 Technology Infrastructure and Metadata 152.1.4 Data Profi ling and Analytics 152.1.5 Data Integration 152.1.6 Data Assessment 162.1.7 Issues Resolution (IR) 162.1.8 Data Quality Monitoring and Control 162.2 The Data Quality Methodology 172.2.1 Establish a Data Quality Program 172.2.2 Conduct a Current-State Analysis 172.2.3 Strengthen Data Quality Capability through Data Quality Projects 182.2.4 Monitor the Ongoing Production Environment and Measure Data Quality Improvement Effectiveness 182.2.5 Detailed Discussion on Establishing the Data Quality Program 182.2.6 Assess the Current State of Data Quality 212.3 Conclusions 223 The DAIC Approach 233.0 Introduction 233.1 Six Sigma Methodologies 233.1.1 Development of Six Sigma Methodologies 253.2 DAIC Approach for Data Quality 283.2.1 The Defi ne Phase 283.2.2 The Assess Phase 313.2.3 The Improve Phase 363.2.4 The Control Phase (Monitor and Measure) 373.3 Conclusions 40Section II Executing a Data Quality Program 4 Quantification of the Impact of Data Quality 434.0 Introduction 434.1 Building a Data Quality Cost Quantifi cation Framework 434.1.1 The Cost Waterfall 444.1.2 Prioritization Matrix 464.1.3 Remediation and Return on Investment 504.2 A Trading Offi ce Illustrative Example 514.3 Conclusions 545 Statistical Process Control and Its Relevance in Data Quality Monitoring and Reporting 555.0 Introduction 555.1 What Is Statistical Process Control? 555.1.1 Common Causes and Special Causes 575.2 Control Charts 595.2.1 Different Types of Data 595.2.2 Sample and Sample Parameters 605.2.3 Construction of Attribute Control Charts 625.2.4 Construction of Variable Control Charts 655.2.5 Other Control Charts 675.2.6 Multivariate Process Control Charts 695.3 Relevance of Statistical Process Control in Data Quality Monitoring and Reporting 695.4 Conclusions 706 Critical Data Elements: Identification, Validation, and Assessment 716.0 Introduction 716.1 Identifi cation of Critical Data Elements 716.1.1 Data Elements and Critical Data Elements 716.1.2 CDE Rationalization Matrix 726.2 Assessment of Critical Data Elements 756.2.1 Data Quality Dimensions 766.2.2 Data Quality Business Rules 786.2.3 Data Profi ling 796.2.4 Measurement of Data Quality Scores 806.2.5 Results Recording and Reporting (Scorecard) 806.3 Conclusions 827 Prioritization of Critical Data Elements (Funnel Approach) 837.0 Introduction 837.1 The Funnel Methodology (Statistical Analysis for CDE Reduction) 837.1.1 Correlation and Regression Analysis for Continuous CDEs 857.1.2 Association Analysis for Discrete CDEs 887.1.3 Signal-to-Noise Ratios Analysis 907.2 Case Study: Basel II 917.2.1 Basel II: CDE Rationalization Matrix 917.2.2 Basel II: Correlation and Regression Analysis 947.2.3 Basel II: Signal-to-Noise (S/N) Ratios 967.3 Conclusions 998 Data Quality Monitoring and Reporting Scorecards 1018.0 Introduction 1018.1 Development of the DQ Scorecards 1028.2 Analytical Framework (ANOVA, SPCs, Thresholds, Heat Maps) 1028.2.1 Thresholds and Heat Maps 1038.2.2 Analysis of Variance (ANOVA) and SPC Charts 1078.3 Application of the Framework 1098.4 Conclusions 1129 Data Quality Issue Resolution 1139.0 Introduction 1139.1 Description of the Methodology 1139.2 Data Quality Methodology 1149.3 Process Quality/Six Sigma Approach 1159.4 Case Study: Issue Resolution Process Reengineering 1179.5 Conclusions 11910 Information System Testing 12110.0 Introduction 12110.1 Typical System Arrangement 12210.1.1 The Role of Orthogonal Arrays 12310.2 Method of System Testing 12310.2.1 Study of Two-Factor Combinations 12310.2.2 Construction of Combination Tables 12410.3 MTS Software Testing 12610.4 Case Study: A Japanese Software Company 13010.5 Case Study: A Finance Company 13310.6 Conclusions 13811 Statistical Approach for Data Tracing 13911.0 Introduction 13911.1 Data Tracing Methodology 13911.1.1 Statistical Sampling 14211.2 Case Study: Tracing 14411.2.1 Analysis of Test Cases and CDE Prioritization 14411.3 Data Lineage through Data Tracing 14911.4 Conclusions 15112 Design and Development of Multivariate Diagnostic Systems 15312.0 Introduction 15312.1 The Mahalanobis-Taguchi Strategy 15312.1.1 The Gram Schmidt Orthogonalization Process 15512.2 Stages in MTS 15812.3 The Role of Orthogonal Arrays and Signal-to-Noise Ratio in Multivariate Diagnosis 15912.3.1 The Role of Orthogonal Arrays 15912.3.2 The Role of S/N Ratios in MTS 16112.3.3 Types of S/N Ratios 16212.3.4 Direction of Abnormals 16412.4 A Medical Diagnosis Example 17212.5 Case Study: Improving Client Experience 17512.5.1 Improvements Made Based on Recommendations from MTS Analysis 17712.6 Case Study: Understanding the Behavior Patterns of Defaulting Customers 17812.7 Case Study: Marketing 18012.7.1 Construction of the Reference Group 18112.7.2 Validation of the Scale 18112.7.3 Identification of Useful Variables 18112.8 Case Study: Gear Motor Assembly 18212.8.1 Apparatus 18312.8.2 Sensors 18412.8.3 High-Resolution Encoder 18412.8.4 Life Test 18512.8.5 Characterization 18512.8.6 Construction of the Reference Group or Mahalanobis Space 18612.8.7 Validation of the MTS Scale 18712.8.8 Selection of Useful Variables 18812.9 Conclusions 18913 Data Analytics 19113.0 Introduction 19113.1 Data and Analytics as Key Resources 19113.1.1 Different Types of Analytics 19313.1.2 Requirements for Executing Analytics 19513.1.3 Process of Executing Analytics 19613.2 Data Innovation 19713.2.1 Big Data 19813.2.2 Big Data Analytics 19913.2.3 Big Data Analytics Operating Model 20613.2.4 Big Data Analytics Projects: Examples 20713.3 Conclusions 20814. Building a Data Quality Practices Center 20914.0 Introduction 20914.1 Building a DQPC 20914.2 Conclusions 211Appendix A 213Equations for Signal-to-Noise (S/N) Ratios 213Nondynamic S/N Ratios 213Dynamic S/N Ratios 214Appendix B 217Matrix Theory: Related Topics 217What Is a Matrix? 217Appendix C 221Some Useful Orthogonal Arrays 221Two-Level Orthogonal Arrays 221Three-Level Orthogonal Arrays 255Index of Terms and Symbols 259References 261Index 267