DevOps Handbook

Gene Kim is a multiple award-winning CTO, researcher and bestselling author. He was founder and CTO of Tripwire for 13 years. He has written six books. Since 2014, he has been the founder of IT Revolution and the organizer of the DevOps Enterprise Summit.Jez Humble is co-author of several books on software including Shingo Publication Award winner Accelerate, Jolt Award winner Continuous Delivery, and The DevOps Handbook. He has spent his career tinkering with code, infrastructure, and product development in companies of varying sizes across three continents. He works for Google Cloud as a technology advocate and teaches at UC Berkeley.Patrick Debois is the Director of DevOps Relations and Advisor at Snyk. In 2009 he coined the word DevOps by organizing the first devopsdays event, as is now often known as one of the grandfathers of DevOps. He organized conferences all over the world to collect and spread new ideas.John Willis is Senior Director of the Global Transformation Office at Red Hat. Prior to Red Hat, he was the Director of Ecosystem Development for Docker. John was one of the earliest cloud evangelists and is considered one of the founders of the DevOps movement. John is the author of 7 IBM Redbooks, as well as co-author of the The DevOps Handbook and Beyond the Phoenix Project.Nicole Forsgren, PhD, is a Partner at Microsoft Research. She is author of the Shingo Publication Award-winning book Accelerate: The Science of Lean Software and DevOps and is best known as lead investigator on the largest DevOps studies to date. She has been a successful entrepreneur (with an exit to Google), professor, performance engineer, and sysadmin. Her work has been published in several peer-reviewed journals.

• CONTENTSFigures & Tables xiiNote from the Publisher on the Second Edition xvForeword to the Second Edition: Nicole Forsgren xixForeword to the First Edition: John Allspaw xxiPreface: Aha! xxiiiIntroduction xxxiPart I—The Three WaysPart I Introduction 301 Agile, Continuous Delivery, and the Three Ways 7NEW Case Study: Approaching Cruising Altitude:American Airlines' DevOps Journey Part 1 (2020) 1502 The First Way: The Principles of Flow 19NEW Case Study: Flow and Constraint Management inHealthcare (2021) 2903 The Second Way: The Principles of Feedback 33NEW Case Study: Pulling the Andon Cord atExcella (2019) 3904 The Third Way: The Principles of Continual Learning andExperimentation 45NEW Case Study: The Story of Bell Labs 54Part 1 Conclusion 57Part II—Where to StartPart II Introduction 6105 Selecting Which Value Stream to Start With 63Case Study: Nordstrom's DevOps Transformation 63NEW Case Study: Kessel Run: The BrownfieldTransformation of a Mid-Air Refueling System (2020) 69NEW Case Study: Scaling DevOps Across the Business:American Airlines' DevOps Journey (Part 2) (2020) 74viii CONTENTSNEW Case Study: Saving the Economy From Ruin (With aHyperscale PaaS) (2021) 7706 Understanding the Work in Our Value Stream, Making itVisible, and Expanding it Across the Organization 81Case Study: Nordstrom's Experience with Value StreamMapping 81Case Study: Operation InVersion at LinkedIn (2011) 9107 How to Design Our Organization and Architecturewith Conway's Law in Mind 97Case Study: Conway's Law at Etsy 98Case Study: API Enablement at Target (2015) 11208 How to Get Great Outcomes by Integrating Operationsinto the Daily Work of Development 115Case Study: Big Fish Games 115NEW Case Study: Better Ways of Working at NationwideBuilding Society 124Part II Conclusion 129Part III—The First Way: The Technical Practices of FlowPart III Introduction 13309 Create the Foundations of Our Deployment Pipeline 135Case Study: Enterprise Data Warehouse 135NEW Case Study: How a Hotel Company Ran $30B ofRevenue in Containers (2020) 14310 Enable Fast and Reliable Automated Testing 147Case Study: Google Web Server 14811 Enable and Practice Continuous Integration 167Case Study: HP LaserJet Firmware 168Case Study: Continuous Integration of Bazaarvoice (2012) 17312 Automate and Enable Low-Risk Releases 177Case Study: Daily Deployments at CSG International (2013) 181Case Study: Etsy—Self-Service Developer Deployment:An Example of Continuous Deployment (2014) 186Case Study: Dixons Retail—Blue-Green Deployment forPoint-of-Sale System (2008) 193Case Study: Dark Launch of Facebook Chat (2008) 198CONTENTS ixNEW Case Study: Creating a Win-Win for Dev & Opsat CSG (2016) 20113 Architect for Low-Risk Releases 207Case Study: Evolutionary Architecture at Amazon (2002) 202Case Study: Strangler Pattern at Blackboard Learn (2011) 215Part III Conclusion 219Part IV—The Second Way: The Technical Practices of FeedbackPart IV Introduction 22314 Create Telemetry to Enable Seeing and Solving Problems 225Case Study: DevOps Transformation at Etsy (2012) 226Case Study: Creating Self-Service Metrics atLinkedIn (2011) 23715 Analyze Telemetry to Better Anticipate Problems andAchieve Goals 245Case Study: Telemetry at Netflix (2012–2020) 245Case Study: Auto-Scaling Capacity at Netflix (2012) 251Case Study: Advanced Anomaly Detection (2014) 25516 Enable Feedback So Development and Operations CanSafely Deploy Code 259Case Study: Right Media (2006) 259Case Study: The Launch and Hand-Off ReadinessReview Google (2010) 26917 Integrate Hypothesis-Driven Development andA/B Testing into Our Daily Work 273Case Study: Hypothesis-Driven Developmentat Intuit, Inc. (2012) 273Case Study: Doubling Revenue Growth through FastRelease Cycle Experimentation at Yahoo! Answers (2010) 27818 Create Review and Coordination Processes to IncreaseQuality of Our Current Work 281Case Study: Peer Review at GitHub (2011) 286NEW Case Study: From Six-Eye Principle to Release atScale at Adidas (2021) 286Case Study: Code Reviews at Google (2010) 290Case Study: Pair Programming Replacing Broken CodeReview Processes at Pivotal Labs (2011) 293Part IV Conclusion 299x CONTENTSPart V—The Third Way: The Technical Practices of Continual Learningand ExperimentationPart V Introduction 30319 Enable and Inject Learning into Daily Work 305Case Study: AWS US-EAST and Netflix (2011) 305NEW Case Study: Turning an Outage into a PowerfulLearning Opportunity at CSG (2020) 31820 Convert Local Discoveries into Global Improvements 321Case Study: Standardizing a New Technology Stackat Etsy (2010) 332NEW Case Study: Crowdsourcing Technology Governance atTarget (2018) 33321 Reserve Time to Create Organizational Learningand Improvement 335Case Study: 30-Day Challenge at Target (2015) 335Case Study: Internal Technology Conferences atNationwide Insurance, Capital One, and Target (2014) 342Part V Conclusion 347Part VI—The Technological Practices of Integrating InformationSecurity, Change Management, and CompliancePart VI Introduction 35122 Information Security Is Everyone's Job Every Day 353Case Study: Static Security Testing at Twitter (2009) 360Case Study: 18F Automating Compliance for the FederalGovernment with Compliance Masonry (2016) 369Case Study: Instrumenting the Environment atEtsy (2010) 373NEW Case Study: Shifting Security Left at FannieMae (2020) 37623 Protecting the Deployment Pipeline, and Integrating intoChange Management and Other Security and ComplianceControls 379Case Study: Automated Infrastructure Changes asStandard Changes at Salesforce.com (2012) 383CONTENTS xiCase Study: PCI Compliance and a Cautionary Tale ofSeparating Duties at Etsy (2014) 385NEW Case Study: Biz and Tech Partnership toward 10 "No FearReleases" Per Day at Capital One (2020) 387Case Study: Proving Compliance in RegulatedEnvironments (2015) 389Case Study: Relying on Production Telemetry forATM Systems 392Part VI Conclusion 395A Call to Action: Conclusion to the DevOps Handbook 397Afterword to the Second Edition 401Appendices 409Bibliography 423Notes 441Index 461Acknowledgments 479Author Biographies 482

The DevOps Handbook has been a critical resource when working with clients to transform their software delivery culture and processes. The book provides easy to understand, practical patterns for improving workflow, communication, and product delivery.