Software Process Dynamics

Raymond J. Madachy, PhD, is a Research Assistant Professor in the USC Industrial and Systems Engineering Department and a Principal of the USC Center for Systems and Software Engineering. Dr. Madachy's current research interests include modeling and simulation of processes for architecting and engineering of complex software-intensive systems; economic analysis and value-based engineering of software-intensive systems; systems and software measurement, process improvement, and quality; quantitative methods for systems risk management; integrating systems engineering and software engineering disciplines; and integrating empirical-based research with process simulation. He is a Senior Member of IEEE and a member of ACM.

• Foreword xiiiBarry BoehmPreface xviiPart 1 FundamentalsChapter 1 Introduction and Background 31.1 Systems, Processes, Models, and Simulation 61.2 Systems Thinking 81.2.1 The Fifth Discipline and Common Models 91.2.2 Systems Thinking Compared to System Dynamics 91.2.3 Weinberg’s Systems Thinking 101.3 Basic Feedback Systems Concepts Applied to the Software Process 101.3.1 Using Simulation Models for Project Feedback 131.3.2 System Dynamics Introductory Example 141.4 Brooks’s Law Example 161.4.1 Brooks’s Law Model Behavior 191.5 Software Process Technology Overview 221.5.1 Software Process Modeling 221.5.2 Process Lifecycle Models 291.5.3 Process Improvement 401.6 Challenges for the Software Industry 451.7 Major References 471.8 Chapter 1 Summary 481.9 Exercises 49Chapter 2 The Modeling Process with System Dynamics 532.1 System Dynamics Background 542.1.1 Conserved Flows Versus Nonconserved Information 552.1.2 The Continuous View Versus Discrete Event Modeling 552.1.3 Model Elements and Notations 562.1.4 Mathematical Formulation of System Dynamics 562.1.5 Using Heuristics 602.1.6 Potential Pitfalls 602.2 General System Behaviors 612.2.1 Goal-Seeking Behavior 612.2.2 Information Smoothing 632.2.3 Example: Basic Structures for General Behaviors 632.3 Modeling Overview 642.3.1 An Iterative Process 682.3.2 Applying the WinWin Spiral Model 702.4 Problem Definition 732.4.1 Defining the Purpose 732.4.2 Reference Behavior 742.4.3 Example: Model Purpose and Reference Behavior 752.5 Model Conceptualization 752.5.1 Identification of System Boundary 782.5.2 Causal Loop Diagrams 792.6 Model Formulation and Construction 832.6.1 Top-Level Formulation 842.6.2 Basic Patterns and Rate Equations 902.6.3 Graph and Table Functions 962.6.4 Assigning Parameter Values 992.6.5 Model Building Principles 1012.6.6 Model Integration 1032.6.7 Example: Construction Iterations 1042.7 Simulation 1102.7.1 Steady-state Conditions 1122.7.2 Test Functions 1132.7.3 Reference Behavior 1152.8 Model Assessment 1162.8.1 Model Validation 1172.8.2 Model Sensitivity Analysis 1212.8.3 Monte Carlo Analysis 1252.9 Policy Analysis 1262.9.1 Policy Parameter Changes 1272.9.2 Policy Structural Changes 1282.9.3 Policy Validity and Robustness 1292.9.4 Policy Suitability and Feasibility 1302.9.5 Example: Policy Analysis 1302.10 Continuous Model Improvement 1312.10.1 Disaggregation 1322.10.2 Feedback Loops 1322.10.3 Hypotheses 1322.10.4 When to Stop? 1332.10.5 Example: Model Improvement Next Steps 1332.11 Software Metrics Considerations 1342.11.1 Data Collection 1342.11.2 Goal–Question–Metric Framework 1352.11.3 Integrated Measurement and Simulation 1362.12 Project Management Considerations 1382.12.1 Modeling Communication and Team Issues 1392.12.2 Risk Management of Modeling Projects 1402.12.3 Modeling Documentation and Presentation 1412.12.4 Modeling Work Breakdown Structure 1422.13 Modeling Tools 1422.14 Major References 1452.15 Chapter 2 Summary 1462.15.1 Summary of Modeling Heuristics 1482.16 Exercises 149Chapter 3 Model Structures and Behaviors for Software Processes 1553.1 Introduction 1553.2 Model Elements 1573.2.1 Levels (Stocks) 1573.2.2 Rates (Flows) 1593.2.3 Auxiliaries 1593.2.4 Connectors and Feedback Loops 1603.3 Generic Flow Processes 1603.3.1 Rate and Level System 1603.3.2 Flow Chain with Multiple Rates and Levels 1613.3.3 Compounding Process 1623.3.4 Draining Process 1633.3.5 Production Process 1633.3.6 Adjustment Process 1633.3.7 Coflow Process 1643.3.8 Split Flow Process 1653.3.9 Cyclic Loop 1653.4 Infrastructures and Behaviors 1663.4.1 Exponential Growth 1663.4.2 S-Shaped Growth and S-Curves 1673.4.3 Delays 1693.4.4 Balancing Feedback 1753.4.5 Oscillation 1773.4.6 Smoothing 1803.4.7 Production and Rework 1823.4.8 Integrated Production Structure 1833.4.9 Personnel Learning Curve 1833.4.10 Rayleigh Curve Generator 1853.4.11 Attribute Tracking 1863.4.12 Attribute Averaging 1873.4.13 Effort Expenditure Instrumentation 1873.4.14 Decision Structures 1883.5 Software Process Chain Infrastructures 1923.5.1 Software Products 1933.5.2 Defects 1963.5.3 People 2003.6 Major References 2033.7 Chapter 3 Summary 2043.8 Exercises 204Part 2 Applications And Future DirectionsIntroduction to Applications Chapters 211Chapter 4 People Applications 2174.1 Introduction 2174.2 Overview of Applications 2214.3 Project Workforce Modeling 2224.3.1 Example: Personnel Sector Model 2224.4 Exhaustion and Burnout 2244.4.1 Example: Exhaustion Model 2244.5 Learning 2274.5.1 Example: Learning Curve Models 2314.6 Team Composition 2344.6.1 Example: Assessing Agile Team Size for a Hybrid Process 2354.7 Other Application Areas 2524.7.1 Motivation 2524.7.2 Personnel Hiring and Retention 2564.7.3 Skills and Capabilities 2604.7.4 Team Communication 2604.7.5 Negotiation and Collaboration 2614.7.6 Simulation for Personnel Training 2634.8 Major References 2654.9 Chapter 4 Summary 2654.10 Exercises 267Chapter 5 Process and Product Applications 2695.1 Introduction 2695.2 Overview of Applications 2735.3 Peer Reviews 2745.3.1 Example: Modeling an Inspection-Based Process 2755.3.2 Example: Inspection Process Data Calibration 2895.4 Global Process Feedback (Software Evolution) 2915.4.1 Example: Software Evolution Progressive and 293Antiregressive Work5.5 Software Reuse 2995.5.1 Example: Reuse and Fourth-Generation Languages 3015.6 Commercial Off-the-Shelf Software (COTS)-Based Systems 3095.6.1 Example: COTS Glue Code Development and COTS 310Integration5.6.2 Example: COTS-Lifespan Model 3175.7 Software Architecting 3195.7.1 Example: Architecture Development During Inception and 319Elaboration5.8 Quality and Defects 3275.8.1 Example: Defect Dynamics 3285.8.2 Example: Defect Removal Techniques and Orthogonal 330Defect Classification5.9 Requirements Volatility 3335.9.1 Example: Software Project Management Simulator 3375.10 Software Process Improvement 3435.10.1 Example: Software Process Improvement Model 3465.10.2 Example: Xerox Adaptation 3545.11 Major References 3625.12 Provided Models 3635.13 Chapter 5 Summary 3635.14 Exercises 364Chapter 6 Project and Organization Applications 3696.1 Introduction 3696.1.1 Organizational Opportunities for Feedback 3716.2 Overview of Applications 3726.3 Integrated Project Modeling 3736.3.1 Example: Integrated Project Dynamics Model 3736.4 Software Business Case Analysis 3956.4.1 Example: Value-Based Product Modeling 3966.5 Personnel Resource Allocation 4116.5.1 Example: Resource Allocation Policy and Contention Models 4116.6 Staffing 4166.6.1 Example: Rayleigh Manpower Distribution Model 4186.6.2 Example: Process Concurrence Modeling 4236.6.3 Integrating Rayleigh Curves, Process Concurrence, and 441Brooks’s Interpretations6.7 Earned Value 4426.7.2 Example: Earned Value Model 4506.8 Major References 4606.9 Provided Models 4606.10 Chapter 6 Summary 4606.11 Exercises 462Chapter 7 Current and Future Directions 4697.1 Introduction 4697.2 Simulation Environments and Tools 4727.2.1 Usability 4737.2.2 Model Analysis 4737.2.3 Artificial Intelligence and Knowledge-Based Simulation 4747.2.4 Networked Simulations 4757.2.5 Training and Game Playing 4757.3 Model Structures and Component-Based Model Development 4767.3.1 Object-Oriented Methods 4787.3.2 Metamodels 4787.4 New and Emerging Trends for Applications 4797.4.1 Distributed Global Development 4807.4.2 User and People-Oriented Focus 4827.4.3 Agile and Hybrid Processes 4827.4.4 Commercial Off-the-Shelf Software 4847.4.5 Open Source Software Development 4867.4.6 Personnel Talent Supply and Demand 4887.5 Model Integration 4897.5.1 Common Unified Models 4897.5.2 Related Disciplines and Business Processes 4907.5.3 Meta-Model Integration 4917.6 Empirical Research and Theory Building 4927.6.1 Empirical Data Collection for Simulation Models 4937.7 Process Mission Control Centers, Analysis, and Training Facilities 4947.8 Chapter 7 Summary 4967.9 Exercises 498Appendix A: Introduction to Statistics of Simulation 501A.1 Risk Analysis and Probability 502A.2 Probability Distributions 503A.2.1 Interpreting Probability Distributions 505A.2.2 Measures of Location, Variability and Symmetry 506A.2.3 Useful Probability Distributions 508A.3 Monte Carlo Analysis 515A.3.1 Inverse Transform 515A.3.2 Example: Monte Carlo Analysis 516A.4 Analysis of Simulation Input 521A.4.1 Goodness-of-Fit Tests 521A.5 Experimental Design 523A.5.1 Example: Experimental Design and Model Response Surface 524A.6 Analysis of Simulation Output 525A.6.1 Confidence Intervals, Sample Size, and Hypothesis Testing 525A.7 Major References 527A.8 Appendix A Summary 527A.9 Exercises 529Appendix B: Annotated System Dynamics Bibliography 531Appendix C: Provided Models 565References 571Index 593

"By taking both a technical and a social approach Raymond Madachy, the author, stimulates the readers interest and makes his book of over 600 pages a very worthwhile title." (Kybernetes, 2008) "When Ed Yourdon says that this is possible the 'best software engineering book' of the year, and possible the decade, one can hardly argue." (Ubiquity, June 10-16, 2008)