Practical Approach to Large-Scale Agile Development, A: How HP Transformed LaserJet FutureSmart Firmware

Today, even the largest development organizations are turning to agile methodologies, seeking major productivity and quality improvements. However, large-scale agile development is difficult, and publicly available case studies have been scarce. Now, three agile pioneers at Hewlett-Packard present a candid, start-to-finish insider’s look at how they’ve succeeded with agile in one of the company’s most mission-critical software environments: firmware for HP LaserJet printers. This book tells the story of an extraordinary experiment and journey. Could agile principles be applied to re-architect an enormous legacy code base? Could agile enable both timely delivery and ongoing innovation? Could it really be applied to 400+ developers distributed across four states, three continents, and four business units? Could it go beyond delivering incremental gains, to meet the stretch goal of 10x developer productivity improvements? It could, and it did—but getting there was not easy. Writing for both managers and technologists, the authors candidly discuss both their successes and failures, presenting actionable lessons for other development organizations, as well as approaches that have proven themselves repeatedly in HP’s challenging environment. They not only illuminate the potential benefits of agile in large-scale development, they also systematically show how these benefits can actually be achieved. Coverage includes: • Tightly linking agile methods and enterprise architecture with business objectives • Focusing agile practices on your worst development pain points to get the most bang for your buck • Abandoning classic agile methods that don’t work at the largest scale • Employing agile methods to establish a new architecture • Using metrics as a “conversation starter” around agile process improvements • Leveraging continuous integration and quality systems to reduce costs, accelerate schedules, and automate the delivery pipeline • Taming the planning beast with “light-touch” agile planning and lightweight long-range forecasting • Implementing effective project management and ensuring accountability in large agile projects • Managing tradeoffs associated with key decisions about organizational structure • Overcoming U.S./India cultural differences that can complicate offshore development • Selecting tools to support quantum leaps in productivity in your organization • Using change management disciplines to support greater enterprise agility

Table of Contents:
Foreword by Jim Highsmith, ThoughtWorks     xiii Preface     xv Chapter 1  Agile Principles versus Practices     1 The Principles of the Agile Manifesto     2 Our Take on Agile/Lean Principles     3 A Quick Tutorial: Agile versus Waterfall     6 Summary     8 Chapter 2  Tuning Agile to Your Business Objectives     9 Background: HP FutureSmart Firmware Case Study     10 Cost and Cycle-Time Drivers Prior to HP FutureSmart Firmware     11 Value Proposition of Re-Architecting the HP FutureSmart Firmware and Processes     13 Establish Development Objectives from the Business Analysis     15 Summary     16 Chapter 3  Aligning Architecture with Business Objectives     17 Challenges with Existing Architecture     18 Architecting for the Business: Dynamic Variability and Forward Compatibility     19 Keeping an Architecture Current and Sustainable     22 Summary     25 Chapter 4  How to Establish a New Architecture Using Agile Concepts     27 Re-Architecting Iteratively     28 Making Progress     28 The Thin-Slice Model     30 Creating Cultural Shifts Through Architectural Demos     31 Summary     33 Chapter 5  The Real Secret to Success in Large-Scale Agile     35 Change for People’s Sake     36 Metrics Are a Conversation Starter     38 Iterative Model of Agile Management     39    Mini-Milestone Objectives     40    Cascading Objectives to Track Progress     41    Conversations     42    Learning     43    Agile Adjustments     44 Summary     44 Chapter 6  Continuous Integration and Quality Systems     45 Reducing Build Resources and Build Time: Continuous Integration     46 Achieving High Quality with CI: Automated Multilevel Testing     55    L0 Testing     57    L1 Testing     58    L2 Testing     58    L3 Testing     59    L4 Testing     60    Continuous Improvement of the Deployment Pipeline     60 Productivity Results of Our Automated Delivery Pipeline     61 Special Considerations for Enterprise Software Systems     63 Summary     65 Chapter 7  Taming the Planning Beast     67 Predict by Ballparking and Trend Watching     69    Ballpark Prediction: R&D Early Response to High-Level Initiatives     70    Trend Watching: Quick Response to All Feature Requestors (Where They’re Likely to Land)     70 Clear Prioritization     73 Just-in-Time User Story Definition     76    Invest in System Engineering     77    Put Marketing in Charge of a Unified 1-N List     80    Involve the Technology Architects     81    Use Project Managers as “Feature Leads”     81    Reuse Requirements and Test Tags for Scalability     82    Commit by Delivering, Not by Estimating     83 Convincing the Business: Agile Planning Is Okay     86 Summary     88 Chapter 8  Unique Challenges of Estimating Large Innovations     91 Waterfall Approach and Challenges     92 Agile Approach     92 Challenging Situations with the Agile Approach: Large Architectural Efforts     95 Change Management and Integrating with the Business     98 Summary     100 Chapter 9  Our Take on Project Management for Large-Scale Agile     101 Oversight and Priority: Program Managers     102 Accountability: Section Managers     103 Robustness and Scalability: Architects     104 Putting It All Together     104 Summary     105 Chapter 10  Organizational Approach: Managing to Disadvantages     107 Test Ownership Organization     108 Component versus Feature Organization     111 Traditionally Managed Project Teams versus Self-Managed Scrum Teams     114 Summary     116 Chapter 11  Effective Agile Development across U.S. and Indian Cultures     117 Lesson 1: Permission to Ask     118 Lesson 2: Ensure Time to Explore     119 Lesson 3: Have Small Wins First     119 Lesson 4: Exploit the Time Difference     120 Lesson 5: Take Time to Train–Always     121 Lesson 6: Remember a Team Is about People     121 Organizing for the Highest Leverage of Offshore Teams     122 Summary     125 Chapter 12  The Right Tools: Quantum Leaps in Productivity     127 Common Development Environment     128 Simulation and Emulation Environment for Automated Testing     129 Test Architecture for Scalability: Common Test Framework (CTF)     131 The Most Important Part of Test Automation: Virtual Machine Provisioning System (VMPS)     133 Real-Time Metrics and Tracking     136 Integrated Toolset     137 Cool Toys Worth Investing In     138 Summary     139 Chapter 13  Real-World Agile Results: HP FutureSmart Firmware     141 Resources Moved from Overhead to Innovation     142 R&D and Developer Productivity     144 Improvement in Current Product Support     146 Summary     147 Chapter 14  Change Management in Moving Toward Enterprise Agility     149 Impacts on Other R&D Groups and System Qualification     150 Impacts on Product Program Teams     151 Impacts on Non-R&D Product Generation Activities/Teams     154 Where to Draw Boundaries with Coordinating Organizational Agility     155 Change Management of the HP FutureSmart Firmware Transformation     156 Summary     158 Chapter 15  Differences in Our Perspective on Scaling Agile     159 A Difference in Perspective     160 Focusing on Agility Rather Than Team Operations     161 Changing the Deployment Pipeline     162 Embracing the Uncertainty of Agile     163 Enterprisewide Tracking and Incremental Improvements     164 Summary     164 Chapter 16  Taking the First Step     167 Figuring Out First Steps     168 What’s Next for FutureSmart?     169 Determining Your First Steps     171 Summary     172 Appendix A  Twelve Principles of Agile Software     173 Bibliography     175 Index     177