Software Process Dynamics

This book is designed for professionals and students in software engineering or information technology who are interested in understanding the dynamics of software development in order to assess and optimize their own process strategies. It explains how simulation of interrelated technical and social factors can provide a means for organizations to vastly improve their processes. It is structured for readers to approach the subject from different perspectives, and includes descriptive summaries of the best research and applications.

Table of Contents:
Foreword xiii Barry Boehm Preface xvii Part 1 Fundamentals Chapter 1 Introduction and Background 3 1.1 Systems, Processes, Models, and Simulation 6 1.2 Systems Thinking 8 1.2.1 The Fifth Discipline and Common Models 9 1.2.2 Systems Thinking Compared to System Dynamics 9 1.2.3 Weinberg’s Systems Thinking 10 1.3 Basic Feedback Systems Concepts Applied to the Software Process 10 1.3.1 Using Simulation Models for Project Feedback 13 1.3.2 System Dynamics Introductory Example 14 1.4 Brooks’s Law Example 16 1.4.1 Brooks’s Law Model Behavior 19 1.5 Software Process Technology Overview 22 1.5.1 Software Process Modeling 22 1.5.2 Process Lifecycle Models 29 1.5.3 Process Improvement 40 1.6 Challenges for the Software Industry 45 1.7 Major References 47 1.8 Chapter 1 Summary 48 1.9 Exercises 49 Chapter 2 The Modeling Process with System Dynamics 53 2.1 System Dynamics Background 54 2.1.1 Conserved Flows Versus Nonconserved Information 55 2.1.2 The Continuous View Versus Discrete Event Modeling 55 2.1.3 Model Elements and Notations 56 2.1.4 Mathematical Formulation of System Dynamics 56 2.1.5 Using Heuristics 60 2.1.6 Potential Pitfalls 60 2.2 General System Behaviors 61 2.2.1 Goal-Seeking Behavior 61 2.2.2 Information Smoothing 63 2.2.3 Example: Basic Structures for General Behaviors 63 2.3 Modeling Overview 64 2.3.1 An Iterative Process 68 2.3.2 Applying the WinWin Spiral Model 70 2.4 Problem Definition 73 2.4.1 Defining the Purpose 73 2.4.2 Reference Behavior 74 2.4.3 Example: Model Purpose and Reference Behavior 75 2.5 Model Conceptualization 75 2.5.1 Identification of System Boundary 78 2.5.2 Causal Loop Diagrams 79 2.6 Model Formulation and Construction 83 2.6.1 Top-Level Formulation 84 2.6.2 Basic Patterns and Rate Equations 90 2.6.3 Graph and Table Functions 96 2.6.4 Assigning Parameter Values 99 2.6.5 Model Building Principles 101 2.6.6 Model Integration 103 2.6.7 Example: Construction Iterations 104 2.7 Simulation 110 2.7.1 Steady-state Conditions 112 2.7.2 Test Functions 113 2.7.3 Reference Behavior 115 2.8 Model Assessment 116 2.8.1 Model Validation 117 2.8.2 Model Sensitivity Analysis 121 2.8.3 Monte Carlo Analysis 125 2.9 Policy Analysis 126 2.9.1 Policy Parameter Changes 127 2.9.2 Policy Structural Changes 128 2.9.3 Policy Validity and Robustness 129 2.9.4 Policy Suitability and Feasibility 130 2.9.5 Example: Policy Analysis 130 2.10 Continuous Model Improvement 131 2.10.1 Disaggregation 132 2.10.2 Feedback Loops 132 2.10.3 Hypotheses 132 2.10.4 When to Stop? 133 2.10.5 Example: Model Improvement Next Steps 133 2.11 Software Metrics Considerations 134 2.11.1 Data Collection 134 2.11.2 Goal–Question–Metric Framework 135 2.11.3 Integrated Measurement and Simulation 136 2.12 Project Management Considerations 138 2.12.1 Modeling Communication and Team Issues 139 2.12.2 Risk Management of Modeling Projects 140 2.12.3 Modeling Documentation and Presentation 141 2.12.4 Modeling Work Breakdown Structure 142 2.13 Modeling Tools 142 2.14 Major References 145 2.15 Chapter 2 Summary 146 2.15.1 Summary of Modeling Heuristics 148 2.16 Exercises 149 Chapter 3 Model Structures and Behaviors for Software Processes 155 3.1 Introduction 155 3.2 Model Elements 157 3.2.1 Levels (Stocks) 157 3.2.2 Rates (Flows) 159 3.2.3 Auxiliaries 159 3.2.4 Connectors and Feedback Loops 160 3.3 Generic Flow Processes 160 3.3.1 Rate and Level System 160 3.3.2 Flow Chain with Multiple Rates and Levels 161 3.3.3 Compounding Process 162 3.3.4 Draining Process 163 3.3.5 Production Process 163 3.3.6 Adjustment Process 163 3.3.7 Coflow Process 164 3.3.8 Split Flow Process 165 3.3.9 Cyclic Loop 165 3.4 Infrastructures and Behaviors 166 3.4.1 Exponential Growth 166 3.4.2 S-Shaped Growth and S-Curves 167 3.4.3 Delays 169 3.4.4 Balancing Feedback 175 3.4.5 Oscillation 177 3.4.6 Smoothing 180 3.4.7 Production and Rework 182 3.4.8 Integrated Production Structure 183 3.4.9 Personnel Learning Curve 183 3.4.10 Rayleigh Curve Generator 185 3.4.11 Attribute Tracking 186 3.4.12 Attribute Averaging 187 3.4.13 Effort Expenditure Instrumentation 187 3.4.14 Decision Structures 188 3.5 Software Process Chain Infrastructures 192 3.5.1 Software Products 193 3.5.2 Defects 196 3.5.3 People 200 3.6 Major References 203 3.7 Chapter 3 Summary 204 3.8 Exercises 204 Part 2 Applications And Future Directions Introduction to Applications Chapters 211 Chapter 4 People Applications 217 4.1 Introduction 217 4.2 Overview of Applications 221 4.3 Project Workforce Modeling 222 4.3.1 Example: Personnel Sector Model 222 4.4 Exhaustion and Burnout 224 4.4.1 Example: Exhaustion Model 224 4.5 Learning 227 4.5.1 Example: Learning Curve Models 231 4.6 Team Composition 234 4.6.1 Example: Assessing Agile Team Size for a Hybrid Process 235 4.7 Other Application Areas 252 4.7.1 Motivation 252 4.7.2 Personnel Hiring and Retention 256 4.7.3 Skills and Capabilities 260 4.7.4 Team Communication 260 4.7.5 Negotiation and Collaboration 261 4.7.6 Simulation for Personnel Training 263 4.8 Major References 265 4.9 Chapter 4 Summary 265 4.10 Exercises 267 Chapter 5 Process and Product Applications 269 5.1 Introduction 269 5.2 Overview of Applications 273 5.3 Peer Reviews 274 5.3.1 Example: Modeling an Inspection-Based Process 275 5.3.2 Example: Inspection Process Data Calibration 289 5.4 Global Process Feedback (Software Evolution) 291 5.4.1 Example: Software Evolution Progressive and 293 Antiregressive Work 5.5 Software Reuse 299 5.5.1 Example: Reuse and Fourth-Generation Languages 301 5.6 Commercial Off-the-Shelf Software (COTS)-Based Systems 309 5.6.1 Example: COTS Glue Code Development and COTS 310 Integration 5.6.2 Example: COTS-Lifespan Model 317 5.7 Software Architecting 319 5.7.1 Example: Architecture Development During Inception and 319 Elaboration 5.8 Quality and Defects 327 5.8.1 Example: Defect Dynamics 328 5.8.2 Example: Defect Removal Techniques and Orthogonal 330 Defect Classification 5.9 Requirements Volatility 333 5.9.1 Example: Software Project Management Simulator 337 5.10 Software Process Improvement 343 5.10.1 Example: Software Process Improvement Model 346 5.10.2 Example: Xerox Adaptation 354 5.11 Major References 362 5.12 Provided Models 363 5.13 Chapter 5 Summary 363 5.14 Exercises 364 Chapter 6 Project and Organization Applications 369 6.1 Introduction 369 6.1.1 Organizational Opportunities for Feedback 371 6.2 Overview of Applications 372 6.3 Integrated Project Modeling 373 6.3.1 Example: Integrated Project Dynamics Model 373 6.4 Software Business Case Analysis 395 6.4.1 Example: Value-Based Product Modeling 396 6.5 Personnel Resource Allocation 411 6.5.1 Example: Resource Allocation Policy and Contention Models 411 6.6 Staffing 416 6.6.1 Example: Rayleigh Manpower Distribution Model 418 6.6.2 Example: Process Concurrence Modeling 423 6.6.3 Integrating Rayleigh Curves, Process Concurrence, and 441 Brooks’s Interpretations 6.7 Earned Value 442 6.7.2 Example: Earned Value Model 450 6.8 Major References 460 6.9 Provided Models 460 6.10 Chapter 6 Summary 460 6.11 Exercises 462 Chapter 7 Current and Future Directions 469 7.1 Introduction 469 7.2 Simulation Environments and Tools 472 7.2.1 Usability 473 7.2.2 Model Analysis 473 7.2.3 Artificial Intelligence and Knowledge-Based Simulation 474 7.2.4 Networked Simulations 475 7.2.5 Training and Game Playing 475 7.3 Model Structures and Component-Based Model Development 476 7.3.1 Object-Oriented Methods 478 7.3.2 Metamodels 478 7.4 New and Emerging Trends for Applications 479 7.4.1 Distributed Global Development 480 7.4.2 User and People-Oriented Focus 482 7.4.3 Agile and Hybrid Processes 482 7.4.4 Commercial Off-the-Shelf Software 484 7.4.5 Open Source Software Development 486 7.4.6 Personnel Talent Supply and Demand 488 7.5 Model Integration 489 7.5.1 Common Unified Models 489 7.5.2 Related Disciplines and Business Processes 490 7.5.3 Meta-Model Integration 491 7.6 Empirical Research and Theory Building 492 7.6.1 Empirical Data Collection for Simulation Models 493 7.7 Process Mission Control Centers, Analysis, and Training Facilities 494 7.8 Chapter 7 Summary 496 7.9 Exercises 498 Appendix A: Introduction to Statistics of Simulation 501 A.1 Risk Analysis and Probability 502 A.2 Probability Distributions 503 A.2.1 Interpreting Probability Distributions 505 A.2.2 Measures of Location, Variability and Symmetry 506 A.2.3 Useful Probability Distributions 508 A.3 Monte Carlo Analysis 515 A.3.1 Inverse Transform 515 A.3.2 Example: Monte Carlo Analysis 516 A.4 Analysis of Simulation Input 521 A.4.1 Goodness-of-Fit Tests 521 A.5 Experimental Design 523 A.5.1 Example: Experimental Design and Model Response Surface 524 A.6 Analysis of Simulation Output 525 A.6.1 Confidence Intervals, Sample Size, and Hypothesis Testing 525 A.7 Major References 527 A.8 Appendix A Summary 527 A.9 Exercises 529 Appendix B: Annotated System Dynamics Bibliography 531 Appendix C: Provided Models 565 References 571 Index 593