Using Agile methods, you can bring far greater innovation, value, and quality to any data warehousing (DW), business intelligence (BI), or analytics project. However, conventional Agile methods must be carefully adapted to address the unique characteristics of DW/BI projects. In Agile Analytics, Agile pioneer Ken Collier shows how to do just that.
Collier introduces platform-agnostic Agile solutions for integrating infrastructures consisting of diverse operational, legacy, and specialty systems that mix commercial and custom code. Using working examples, he shows how to manage analytics development teams with widely diverse skill sets and how to support enormous and fast-growing data volumes. Collier’s techniques offer optimal value whether your projects involve “back-end” data management, “front-end” business analysis, or both.
- Part I focuses on Agile project management techniques and delivery team coordination, introducing core practices that shape the way your Agile DW/BI project community can collaborate toward success
-
- Part II presents technical methods for enabling continuous delivery of business value at production-quality levels, including evolving superior designs; test-driven DW development; version control; and project automation
-
Collier brings together proven solutions you can apply right now—whether you’re an IT decision-maker, data warehouse professional, database administrator, business intelligence specialist, or database developer. With his help, you can mitigate project risk, improve business alignment, achieve better results—and have fun along the way.
Table of Contents:
Foreword by Jim Highsmith xv Foreword by Wayne Eckerson xvii
Preface xix
Acknowledgments xxxiii
About the Author xxxv
Part I: Agile Analytics: Management Methods 1
Chapter 1: Introducing Agile Analytics 3
Alpine-Style Systems Development 4
What Is Agile Analytics? 7
Data Warehousing Architectures and Skill Sets 13
Why Do We Need Agile Analytics? 16
Introducing FlixBuster Analytics 22
Wrap-Up 23
Chapter 2: Agile Project Management 25
What Is Agile Project Management? 26
Phased-Sequential DW/BI Development 30
Envision → Explore Instead of Plan → Do 32
Changing the Role of Project Management 35
Making Sense of Agile “Flavors” 36
Tenets of Agility 39
Wrap-Up 56
Chapter 3: Community, Customers, and Collaboration 59
What Are Agile Community and Collaboration? 60
The Agile Community 64
A Continuum of Trust 67
The Mechanics of Collaboration 69
Consumer Collaboration 73
Doer Collaboration 77
Planner Collaboration 78
Precursors to Agility 80
Wrap-Up 82
Chapter 4: User Stories for BI Systems 85
What Are User Stories? 86
User Stories versus Requirements 89
From Roles to Use Cases to User Stories 92
Decomposing Epics 99
What’s the Smallest, Simplest Thing? 103
Story Prioritization and Backlog Management 107
Story-Point Estimating 111
Parking Lot Diagrams 117
Wrap-Up 119
Chapter 5: Self-Organizing Teams Boost Performance 121
What Is a Self-Organizing Team? 122
Self-Organization Requires Self-Discipline 127
Self-Organization Requires Shared Responsibility 128
Self-Organization Requires Team Working Agreements 130
Self-Organization Requires Honoring Commitments 132
Self-Organization Requires Glass-House Development 134
Self-Organizing Requires Corporate Alignment 136
Wrap-Up 137
Part II: Agile Analytics: Technical Methods 139
Chapter 6: Evolving Excellent Design 141
What Is Evolutionary Design? 144
How Much Up-Front Design? 148
Agile Modeling 149
Data Model Patterns 152
Managing Technical Debt 154
Refactoring 157
What Is Refactoring? 159
Deploying Warehouse Changes 167
Other Reasons to Take an Evolutionary Approach 171
Case Study: Adaptive Warehouse Architecture 174
Wrap-Up 189
Chapter 7: Test-Driven Data Warehouse Development 193
What Is Agile Analytics Testing? 194
Agile Testing Framework 197
BI Test Automation 201
Sandbox Development 211
Test-First BI Development 215
BI Testing Guidelines 220
Setup Time 221
Functional BI Testing 222
Wrap-Up 223
Chapter 8: Version Control for Data Warehousing 225
What Is Version Control? 226
The Repository 230
Working with Files 233
Organizing the Repository 240
Tagging and Branching 245
Choosing an Effective Tool 252
Wrap-Up 254
Chapter 9: Project Automation 257
What Is Project Automation? 258
Getting Started 261
Build Automation 262
Continuous Integration 274
Push-Button Releases 281
Wrap-Up 288
Chapter 10: Final Words 291
Focus on the Real Problem 291
Being Agile versus Doing Agile 293
Gnarly Problems 296
What about Emerging Technologies? 298
Adoption Strategies 299
Closing Thoughts . . . 306
References and Recommended Reading 309
Index 315
About the Author :
Ken Collier has worked with Agile methods since 2003, and pioneered the integration of Agile methods with data warehousing, business intelligence, and analytics to create the Agile Analytics style. He continues to refine these ideas as technical lead and project manager on several Agile DW/BI project teams. Collier frequently trains DW/BI teams in Agile Analytics, and has been a keynote speaker on the subject at HEDW (Higher Education Data Warehouse) 2011 and multiple TDWI (The Data Warehousing Institute) World Conferences. He is founder and president of KWC Technologies, Inc., and a senior consultant in the Cutter Consortium’s Agile Development and Business Intelligence practice areas.
