About the Book
Successfully Implement High-Value Configuration Management Processes in Any Development Environment
As IT systems have grown increasingly complex and mission-critical, effective configuration management (CM) has become critical to an organization’s success. Using CM best practices, IT professionals can systematically manage change, avoiding unexpected problems introduced by changes to hardware, software, or networks. Now, today’s best CM practices have been gathered in one indispensable resource showing you how to implement them throughout any agile or traditional development organization.
Configuration Management Best Practices is practical, easy to understand and apply, and fully reflects the day-to-day realities faced by practitioners. Bob Aiello and Leslie Sachs thoroughly address all six “pillars” of CM: source code management, build engineering, environment configuration, change control, release engineering, and deployment. They demonstrate how to implement CM in ways that support software and systems development, meet compliance rules such as SOX and SAS-70, anticipate emerging standards such as IEEE/ISO 12207, and integrate with modern frameworks such as ITIL, COBIT, and CMMI. Coverage includes
Using CM to meet business objectives, contractual requirements, and compliance rules
Enhancing quality and productivity through lean processes and “just-in-time” process improvement
Getting off to a good start in organizations without effective CM
Implementing a Core CM Best Practices Framework that supports the entire development lifecycle
Mastering the “people” side of CM: rightsizing processes, overcoming resistance, and understanding
workplace psychology
Architecting applications to take full advantage of CM best practices
Establishing effective IT controls and compliance
Managing tradeoffs and costs and avoiding expensive pitfalls
Configuration Management Best Practices is the essential resource for everyone concerned with CM: from CTOs and CIOs to development, QA, and project managers and software engineers to analysts, testers, and compliance professionals.
Praise for Configuration Management Best Practices
“Understanding change is critical to any attempt to manage change. Bob Aiello and Leslie Sachs’s Configuration Management Best Practices presents fundamental definitions and explanations to help practitioners understand change and its potential impact.”
–Mary Lou A. Hines Fritts, CIO and Vice Provost Academic Programs, University of Missouri-Kansas City
“Few books on software configuration management emphasize the role of people and organizational context in defining and executing an effective SCM process. Bob Aiello and Leslie Sachs’s book will give you the information you need not only to manage change effectively but also to manage the transition to a better SCM process.”
–Steve Berczuk, Agile Software Developer, and author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration
“Bob Aiello and Leslie Sachs succeed handsomely in producing an important book, at a practical and balanced level of detail, for this topic that often ‘goes without saying’ (and hence gets many projects into deep trouble). Their passion for the topic shows as they cover a wonderful range of topics–even culture, personality, and dealing with resistance to change–in an accessible form that can be applied to any project. The software industry has needed a book like this for a long time!”
–Jim Brosseau, Clarrus Consulting Group, and author of Software Teamwork: Taking Ownership for Success
“A must read for anyone developing or managing software or hardware projects. Bob Aiello and Leslie Sachs are able to bridge the language gap between the myriad of communities involved with successful Configuration Management implementations. They describe practical, real world practices that can be implemented by developers, managers, standard makers, and even Classical CM Folk.”
–Bob Ventimiglia, Bobev Consulting
“A fresh and smart review of today’s key concepts of SCM, build management, and related key practices on day-to-day software engineering. From the voice of an expert, Bob Aiello and Leslie Sachs offer an invaluable resource to success in SCM.”
–Pablo Santos Luaces, CEO of Codice Software
“Bob Aiello and Leslie Sachs have a gift for stimulating the types of conversation and thought that necessarily precede needed organizational change. What they have to say is always interesting and often important.”
–Marianne Bays, Business Consultant, Manager and Educator
Table of Contents:
Preface xxi
Introduction xxxiii
PART I THE CORE CM BEST PRACTICES FRAMEWORK 1
Chapter 1 Source Code Management 3
Terminology and Source Code Management 5
Goals of Source Code Management 5
Principles of Source Code Management 6
1.1 Why Is Source Code Management Important? 6
1.2 Where Do I Start? 7
1.3 Source Code Management Core Concepts 9
1.3.1 Creating Baselines and Time Machines 9
1.3.2 Reserved Versus Unreserved Checkouts 10
1.3.3 Sandboxes and Workspaces 11
1.3.4 Variant Management (Branching) 11
1.3.5 Copybranches Versus Deltas 12
1.3.6 How to Handle Bugfixes 12
1.3.7 Streams 14
1.3.8 Merging 15
1.3.9 Changesets 16
1.4 Defect and Requirements Tracking 16
1.5 Managing the Globally Distributed Development Team 17
1.6 Tools Selection 19
1.6.1 Open Source Versus Commercial 21
1.6.2 Product Maturity and Vendor Commitment 21
1.6.3 Extensibility and Open API 22
1.6.4 Don’t Overengineer Your Source Code Management 22
1.7 Recognizing the Cost of Quality (and Total Cost of Ownership) 23
1.7.1 Building Your Source Code Management Budget 24
1.8 Training 24
1.8.1 The “Bob Method” for Training 24
1.9 Defining the Usage Model 25
1.10 Time to Implement and Risks to Success 26
1.11 Establishing Your Support Process 26
1.12 Advanced Features and Empowering Users 27
Conclusion 27
Chapter 2 Build Engineering 29
Goals of Build Engineering 30
Principles of Build Engineering 30
2.1 Why Is Build Engineering Important? 31
2.2 Where Do I Start? 32
2.3 Build Engineering Core Concepts 32
2.3.1 Version IDs or Branding Your Executables 32
2.3.2 Immutable Version IDs 33
2.3.3 Stamping In a Version Label or Tag 33
2.3.4 Managing Compile Dependencies 33
2.3.5 The Independent Build 34
2.4 Core Considerations for Scaling the Build Function 34
2.4.1 Selling the Independent Build 35
2.4.2 Overengineering the Build 35
2.4.3 Testing Your Own Integrity 36
2.4.4 Reporting to Development Can Be a Conflict of Interest 37
2.4.5 Organizational Choices 37
2.5 Build Tools Evaluation and Selection 38
2.5.1 Apache Ant Enters the Build Scene 38
2.5.2 Of Mavens and Other Experts 38
2.5.3 Maven Versus Ant 39
2.5.4 Using Ant for Complex Builds 39
2.5.5 Continuous Integration 40
2.5.6 CI Servers 40
2.5.7 Integrated Development Environments 40
2.5.8 Static Code Analysis 41
2.5.9 Build Frameworks 41
2.5.10 Selecting Your Build Tools 41
2.5.11 Conducting the Bakeoff and Reaching Consensus 42
2.6 Cost of Quality and Training 42
2.7 Making a Good Build Better 42
2.7.1 “Bob-Proofing” Your Build 43
2.7.2 Test-Driven Builds 43
2.7.3 Trust, But Verify 43
2.7.4 The Cockpit of a Plane 44
2.8 The Role of the Build Engineer 44
2.8.1 Know What You Build 45
2.8.2 Partner with Developers 46
2.8.3 Drafting a Rookie 46
2.9 Architecture Is Fundamental 46
2.10 Establishing a Build Process 47
2.10.1 Establishing Organizational Standards 47
2.11 Continuous Integration Versus the Nightly Build 47
2.12 The Future of Build Engineering 48
Conclusion 48
Chapter 3 Environment Configuration 49
Goals of Environment Configuration Control 50
Principles of Environment Configuration Control 51
3.1 Why Is Environment Configuration Important? 51
3.2 Where Do I Start? 51
3.3 Supporting Code Promotion 52
3.4 Managing the Configuration 52
3.4.1 Which Database Are You Using? 53
3.4.2 Did That Trade Go Through? 53
3.4.3 How About a Few Tokens? 54
3.4.4 Centralizing the Environment Variable Assignment 55
3.5 Practical Approaches to Establishing a CMDB 55
3.5.1 Identify and Then Control 56
3.5.2 Understanding the Environment Configuration 56
3.6 Change Control Depends on Environment Configuration 56
3.7 Minimize the Number of Controls Required 57
3.8 Managing Environments 57
3.9 The Future of Environment Configuration 57
Conclusion 58
Chapter 4 Change Control 59
Goals of Change Control 60
Principles of Change Control 60
4.1 Why Is Change Control Important? 61
4.2 Where Do I Start? 61
4.3 The Seven Types of Change Control 61
4.3.1 A Priori 62
4.3.2 Gatekeeping 62
4.3.3 Configuration Control 62
4.3.4 Change Advisory Board 63
4.3.5 Emergency Change Control 64
4.3.6 Process Engineering 64
4.3.7 Senior Management Oversight 64
4.4 Creating a Change Control Function 65
4.5 Examples of Change Control in Action 65
4.5.1 The 29-Minute Change Control Meeting 66
4.5.2 Change Control at the Investment Bank 66
4.5.3 Change Control at the Trading Firm 67
4.5.4 Forging Approvals 69
4.6 Don’t Forget the Risk 69
4.7 Driving the CM Process Through Change Control 69
4.8 Entry/Exit Criteria 70
4.9 After-Action Review 71
4.10 Make Sure That You Evaluate Yourself 71
Conclusion 71
Chapter 5 Release Management 73
Goals of Release Management 74
Principles of Release Management 74
5.1 Why Is Release Management Important? 75
5.2 Where Do I Start? 75
5.3 Release Management Concepts and Practices 76
5.3.1 Packaging Strategies That Work 76
5.3.2 Package Version Identification 76
5.3.3 Sending a Release Map with the Release 77
5.3.4 What Does Immutable Mean? 77
5.4 The Ergonomics of Release Management 77
5.4.1 Avoiding Human Error 78
5.4.2 Understanding the Technology 78
5.4.3 Tools from Build Engineering 79
5.4.4 Avoiding Human Error 79
5.4.5 My Own Three-Step Process 79
5.4.6 Too Many Moving Parts 80
5.5 Release Management as Coordination 80
5.5.1 Communicating the Status of a Release 80
5.5.2 Don’t Forget the Release Calendar 80
5.5.3 RM and Configuration Control 81
5.6 Requirements Tracking 81
5.7 Taking Release Management to the Next Level 81
5.7.1 Using Cryptography to Sign Your Code 82
5.7.2 Operating Systems Support for Release Management 82
5.7.3 Improving Your RM Process 2
Conclusion 83
Chapter 6 Deployment 85
Goals of Deployment 86
Principles of Deployment 86
6.1 Why Is Deployment Important? 87
6.2 Where Do I Start? 87
6.3 Practices and Examples 87
6.3.1 Staging Is Key 87
6.3.2 Scripting the Release Process Itself 89
6.3.3 Frameworks for Deployment 89
6.3.4 What If Bob Makes a Mistake? 89
6.3.5 More on the Depot 90
6.3.6 Auditing Your Release 90
6.4 Conducting a Configuration Audit 91
6.5 Don’t Forget the Smoke Test 92
6.6 Little Things Matter a Lot 92
6.7 Communications Planning 92
6.7.1 Announcing Outages and Completed Deployments 93
6.8 Deployment Should Be Delegated 93
6.9 Trust But Verify 93
6.10 Improving the Deployment Process 93
Conclusion 94
PART II ARCHITECTURE AND HARDWARE CM 95
Chapter 7 Architecting Your Application for CM 97
Goals of Architecting Your Application for CM 98
7.1 Why Is Architecture Important? 99
7.2 Where Do I Start? 99
7.3 How CM Facilitates Good Architecture 99
7.4 What Architects Can Learn From Testers 99
7.4.1 Testing as a Service to the Developers 100
7.5 Configuration Management—Driven Development (CMDD) 101
7.6 Coping with the Changing Architecture 101
7.7 Using Source Code Management to Facilitate Architecture 102
7.8 Training Is Essential 102
7.9 Source Code Management as a Service 103
7.10 Build Engineering as a Service 103
Conclusion 103
Chapter 8 Hardware Configuration Management 105
Goals of Hardware CM 106
8.1 Why Is Hardware CM Important? 106
8.2 Where Do I Start? 107
8.3 When You Can’t Version Control a Circuit Chip 107
8.3.1 A Configuration Item by Any Other Name 107
8.3.2 Version Control for Design Specifications 108
8.4 Don’t Forget the Interfaces 108
8.5 Understanding Dependencies 108
8.6 Traceability 108
8.7 Deploying Changes to the Firmware 109
8.8 The Future of Hardware CM 109
Conclusion 109
PART III THE PEOPLE SIDE OF CM 111
Chapter 9 Rightsizing Your Processes 113
Goals of Rightsizing Your CM Processes 114
9.1 Why Is Rightsizing Your Processes Important? 115
9.2 Where Do I Start? 115
9.3 Verbose Processes Just Get in the Way 116
9.4 SPINs and Promoting the CMM 117
9.5 Disappearing Verbose Processes 117
9.5.1 Agile Processes Just Work 118
9.5.2 Open Unified Process 118
9.5.3 Getting Lean 119
9.5.4 An Extremely Brief Description That I Hope Motivates You to Take a Closer Look at Lean Software Development 119
9.6 The Danger of Having Too Little Process 120
9.7 Just-in-Time Process Improvement 120
9.8 Don’t Overengineer Your CM 120
9.9 Don’t Forget the Technology 121
9.10 Testing Your Own Processes 121
9.11 Process Consultation 122
9.11.1 Transparency That Is Genuine 122
9.12 Create a Structure for Sustainability 122
Conclusion 123
Chapter 10 Overcoming Resistance to Change 125
Goals of Overcoming Resistance to Change 126
10.1 Why Is Overcoming Resistance to Change Important? 127
10.2 Where Do I Start? 127
10.3 Matching Process to Culture 127
10.4 Mixing Psychology and Computer Programming 129
10.5 Process Improvement from Within 129
10.6 Picking Your Battles 131
10.7 Fostering Teamwork 131
10.8 Why Good Developers Oppose Process Improvement 132
10.9 Procedural Justice 132
10.10 Input from Everyone 132
10.11 Showing Leadership 133
10.12 Process Improvement People May Be the Problem 133
10.13 Combining Process and Technology Training 134
10.14 Listening to the Rhythm 135
10.15 Processes Need to Be Tested 136
10.16 Baby Steps and Process Improvement 136
10.17 Selling Process Improvement 137
10.18 What’s in It for Me? 137
10.19 Process Improvement as a Service 137
10.20 Guerrilla Tactics for Process Improvement 138
Conclusion 139
Chapter 11 Personality and CM: A Psychologist Looks at the Workplace 141
Goals of Understanding Personality: What’s in It for Me? 142
11.1 Personality Primer for CM Professionals 144
11.2 What Do CM Experts Need to Consider in Terms of Personality? 146
11.2.1 Communication Styles 147
11.2.2 Do Men and Women Use and Interpret Language Differently? 147
11.2.3 Effective Consultation 148
11.2.4 Verifying the Message 148
11.2.5 Information Processing Preferences 149
11.2.6 Birth Order at Work 150
11.2.7 Firstborns as Leaders 150
11.2.8 The Middle-Born Compromiser 151
11.2.9 The Youngest as Initiator 151
11.2.10 The Only Child 151
11.2.11 Being Yourself 152
11.3 Applying Psychology to the Workplace 152
11.3.1 Effective Teamwork Begins at Home 153
11.3.2 Volleyball or Effective Collaboration 153
11.3.3 Embedding Build Engineers and Testers in the Development Team 153
11.3.4 Blackbox Versus Whitebox Versus Graybox 154
11.3.5 Group Dynamics That Can Damage the Organization 154
11.3.6 Where CM and QA Fit In 154
11.4 Family Dynamics! 155
11.4.1 Indecisiveness 155
11.5 Workplace Culture and Personality 156
11.5.1 Personality and Structure 156
11.5.2 We Already Invented All the Good Ideas 157
11.5.3 Loose Cannons Who Don’t Want to Comply 157
11.5.4 Enforcing Process, While Still Keeping the Train Moving 158
11.5.5 Formulas for Success 158
11.5.6 Caveats 159
Conclusion 159
Chapter 12 Learning From Mistakes That I Have Made 161
Goals of Learning from Mistakes 162
12.1 Why Is It Important to Learn from Our Mistakes? 162
12.2 Where Do I Get Started? 162
12.3 Understanding Our Mistakes 163
12.4 The Mistakes I Have Made 163
12.4.1 Missing the Big Picture 163
12.4.2 Writing Release Automation Can Be Challenging . 164
12.4.3 Thinking That a Good Process Will Carry Itself 165
12.4.4 Failing to Gain Consensus 165
12.4.5 Failing to Show Leadership for CM 165
12.4.6 Becoming Part of the Problem 165
12.4.7 Forgetting to Ask for Help 166
12.5 Turning a Mistake into a Lesson Learned 166
12.5.1 Clarifying What I Need to Get the Job Done 166
12.5.2 Getting the Training That I Need 167
12.6 Common Mistakes That I Have Seen Others Make 167
12.6.1 Ivory Tower 167
12.6.2 Failing to Get Technical and Hands-On 167
12.6.3 Not Being Honest and Open 168
Conclusion 168
PART IV COMPLIANCE, STANDARDS, AND FRAMEWORKS 169
Chapter 13 Establishing IT Controls and Compliance 171
Goals of Establishing IT Controls and Compliance 172
13.1 Why Are IT Controls and Compliance Important? 173
13.2 How Do I Get Started? 173
13.3 Understanding IT Controls and Compliance 174
13.3.1 Sarbanes-Oxley Act of 2002 174
13.3.2 Management Assessment of Internal Controls 174
13.3.3 Committee of Sponsoring Organizations 175
13.3.4 Cobit as a Framework for IT Controls 176
13.3.5 What Does It Mean to Attest to And Report on the Assessment Made by the Management? 176
13.3.6 Health Insurance Portability and Accountability Act of 1996 177
13.3.7 When the GAO Comes Knocking 177
13.3.8 Results of the Audit 178
13.3.9 GAO Reports on NARA’s Configuration Management Practices 179
13.3.10 ERA Configuration Management Plan 179
13.3.11 Areas for Improvement 180
13.3.12 Understanding the Results of the Audit 180
13.3.13 Office of the Comptroller of the Currency 181
13.4 Essential Compliance Requirements 181
13.4.1 Providing Traceability of Requirements to Releases 182
13.4.2 Production Separation of Controls 182
13.5 The Moral Argument for Supporting CM Best Practices 182
13.6 Improving Quality and Productivity Through Compliance 183
13.7 Conducting a CM Assessment 183
13.7.1 Assessment First Steps 184
13.7.2 Listen First Regardless of How Bad the Situation Appears 184
Conclusion 185
Chapter 14 Industry Standards and Frameworks 187
Goals of Using Industry Standards and Frameworks 188
14.1 Why Are Standards and Frameworks Important? 188
14.2 How Do I Get Started? 189
14.3 Terminology Required 189
14.3.1 Configuration Item 189
14.3.2 Configuration Identification 190
14.3.3 Configuration Control 190
14.3.4 Interface Control 190
14.3.5 Configuration Status Accounting 191
14.3.6 Configuration Audit 191
14.3.7 Subcontractor/Vendor Control 192
14.3.8 Conformance Versus Noncompliance 192
14.4 Applying These Terms to the Standards and Frameworks 193
14.5 Industry Standards 193
14.5.1 IEEE 828–Standard for Software Configuration Management Plans 193
14.5.2 ISO 10007–Quality Management Systems–Guidelines for Configuration Management 195
14.5.3 ANSI/ITAA EIA-649-A–National Consensus Standard for Configuration Management 196
14.5.4 ISO/IEC/IEEE 12207 and 15288 196
14.6 Industry Frameworks 196
14.6.1 ISACA Cobit 197
14.6.2 CMM/CMMI 207
14.6.3 itSMF’s ITIL Framework 208
14.6.4 SWEBOK 214
14.6.5 Open Unified Process (OpenUP) 215
14.6.6 Agile/SCRUM 216
Conclusion 217
Index 219
About the Author :
Bob Aiello is the editor-in-chief for CM Crossroads and a consultant specializing in software process improvement, including software configuration and release management. Mr. Aiello has more than 25 years of experience as a technical manager in several top NYC financial services firms where he had companywide responsibility for CM, often providing hands-on technical support for enterprise source code management tools, SOX/Cobit compliance, build engineering, continuous integration, and automated application deployment. Mr. Aiello is the vice chair of the IEEE 828 Standards working group (CM Planning) and is a member of the IEEE Software and Systems Engineering Standards Committee (S2ESC) management board. He is a longstanding member of the steering committee of the NYC Software Process Improvement Network (CitySPIN), where he has served as the chair of the CM SIG. Mr. Aiello holds a master’s degree in industrial psychology from NYU and a bachelor’s degree in computer science and math from Hofstra University.
Leslie Sachs is the COO of Yellow Spider, Inc., which specializes in providing CM-related consulting services that are aligned with the practices described in this book. Ms. Sachs also writes about applying personality to technology endeavors in her column titled Personality Matters. A New York State Certified School Psychologist with more than 20 years of experience, Ms. Sachs has worked in a variety of clinical and business settings where she has provided many effective interventions designed to improve the social and educational functioning of both individuals and groups. Ms. Sachs has a Masters of Science degree in school and community psychology from Pace University and interned in Bellevue Hospital’s famed Psychiatric Center in NYC. A firm believer in the uniqueness of every individual, she has recently done advanced training with Mel Levine’s All Kinds of Minds Institute.
