Modern Software Engineering

David Farley, founder and consultant for Continuous Delivery Ltd., has been a programmer, software engineer, and systems architect since the early days of modern computing. He has shaped ground-breaking approaches that have transformed how developers and teams work. With Jez Humble, Farley coauthored the best-seller Continuous Delivery. As Head of Software Development for the London Multi Asset Exchange (LMAX), he built one of the world's fastest financial exchanges. One of the earliest adopters of agile techniques employing iterative development, continuous integration, and high levels of automated testing, he also coauthored the Reactive Manifesto. Dave also runs the popular and highly successful "Continuous Delivery" YouTube channel.

• Foreword xviiPreface xxiAcknowledgments xxvAbout the Author xxvii Part I: What Is Software Engineering? 1 Chapter 1: Introduction 3 Engineering--The Practical Application of Science 3 What Is Software Engineering? 4 Reclaiming "Software Engineering" 5 The Birth of Software Engineering 7 Shifting the Paradigm 8 Summary 9 Chapter 2: What Is Engineering? 11 Production Is Not Our Problem 11 Design Engineering, Not Production Engineering 12 A Working Definition of Engineering 17 Engineering != Code 17 Why Does Engineering Matter? 19 The Limits of "Craft" 19 Precision and Scalability 20 Managing Complexity 21 Repeatability and Accuracy of Measurement 22 Engineering, Creativity, and Craft 24 Why What We Do Is Not Software Engineering 25 Trade-Offs 26 The Illusion of Progress 26 The Journey from Craft to Engineering 27 Craft Is Not Enough 28 Time for a Rethink? 28 Summary 30 Chapter 3: Fundamentals of an Engineering Approach 31 An Industry of Change? 31 The Importance of Measurement 32 Applying Stability and Throughput 34 The Foundations of a Software Engineering Discipline 36 Experts at Learning 36 Experts at Managing Complexity 37 Summary 38 Part II: Optimize for Learning 41 Chapter 4: Working Iteratively 43 Practical Advantages of Working Iteratively 45 Iteration as a Defensive Design Strategy 46 The Lure of the Plan 48 Practicalities of Working Iteratively 54 Summary 55 Chapter 5: Feedback 57 A Practical Example of the Importance of Feedback 58 Feedback in Coding 60 Feedback in Integration 61 Feedback in Design 63 Feedback in Architecture 65 Prefer Early Feedback 67 Feedback in Product Design 68 Feedback in Organization and Culture 68 Summary 70 Chapter 6: Incrementalism 71 Importance of Modularity 72 Organizational Incrementalism 73 Tools of Incrementalism 74 Limiting the Impact of Change 76 Incremental Design 77 Summary 79 Chapter 7: Empiricism 81 Grounded in Reality 82 Separating Empirical from Experimental 82 "I Know That Bug!" 82 Avoiding Self-Deception 84 Inventing a Reality to Suit Our Argument 85 Guided by Reality 88 Summary 89 Chapter 8: Being Experimental 91 What Does "Being Experimental" Mean? 92 Feedback 93 Hypothesis 94 Measurement 95 Controlling the Variables 96 Automated Testing as Experiments 97 Putting the Experimental Results of Testing into Context 98 Scope of an Experiment 100 Summary 100 Part III: Optimize for Managing Complexity 103 Chapter 9: Modularity 105 Hallmarks of Modularity 106 Undervaluing the Importance of Good Design 107 The Importance of Testability 108 Designing for Testability Improves Modularity 109 Services and Modularity 115 Deployability and Modularity 116 Modularity at Different Scales 118 Modularity in Human Systems 118 Summary 120 Chapter 10: Cohesion 121 Modularity and Cohesion: Fundamentals of Design 121 A Basic Reduction in Cohesion 122 Context Matters 125 High-Performance Software 128 Link to Coupling 129 Driving High Cohesion with TDD 129 How to Achieve Cohesive Software 130 Costs of Poor Cohesion 132 Cohesion in Human Systems 133 Summary 133 Chapter 11: Separation of Concerns 135 Dependency Injection 139 Separating Essential and Accidental Complexity 139 Importance of DDD 142 Testability 144 Ports & Adapters 145 When to Adopt Ports & Adapters 147 What Is an API? 148 Using TDD to Drive Separation of Concerns 149 Summary 150 Chapter 12: Information Hiding and Abstraction 151 Abstraction or Information Hiding 151 What Causes "Big Balls of Mud"? 152 Organizational and Cultural Problems 152 Technical Problems and Problems of Design 154 Fear of Over-Engineering 157 Improving Abstraction Through Testing 159 Power of Abstraction 160 Leaky Abstractions 162 Picking Appropriate Abstractions 163 Abstractions from the Problem Domain 165 Abstract Accidental Complexity 166 Isolate Third-Party Systems and Code 168 Always Prefer to Hide Information 169 Summary 170 Chapter 13: Managing Coupling 171 Cost of Coupling 171 Scaling Up 172 Microservices 173 Decoupling May Mean More Code 175 Loose Coupling Isn't the Only Kind That Matters 176 Prefer Loose Coupling 177 How Does This Differ from Separation of Concerns? 178 DRY Is Too Simplistic 179 Async as a Tool for Loose Coupling 180 Designing for Loose Coupling 182 Loose Coupling in Human Systems 182 Summary 184 Part IV: Tools to Support Engineering in Software 185 Chapter 14: The Tools of an Engineering Discipline 187 What Is Software Development? 188 Testability as a Tool 189 Measurement Points 192 Problems with Achieving Testability 193 How to Improve Testability 196 Deployability 197 Speed 199 Controlling the Variables 200 Continuous Delivery 201 General Tools to Support Engineering 202 Summary 203 Chapter 15: The Modern Software Engineer 205 Engineering as a Human Process 207 Digitally Disruptive Organizations 207 Outcomes vs. Mechanisms 210 Durable and Generally Applicable 211 Foundations of an Engineering Discipline 214 Summary 215 Index 217

" Modern Software Engineering gets it right and describes the ways skilled practitioners actually engineer software today. The techniques Farley presents are not rigid, prescriptive, or linear, but they are disciplined in exactly the ways software requires: empirical, iterative, feedback-driven, economical, and focused on running code." -- Glenn Vanderburg, Director of Engineering at Nubank "There are lots of books that will tell you how to follow a particular software engineering practice; this book is different. What Dave does here is set out the very essence of what defines software engineering and how that is distinct from simple craft. He explains why and how in order to master software engineering you must become a master of both learning and of managing complexity, how practices that already exist support that, and how to judge other ideas on their software engineering merits. This is a book for anyone serious about treating software development as a true engineering discipline, whether you are just starting out or have been building software for decades." -- Dave Hounslow, Software Engineer "These are important topics and it's great to have a compendium that brings them together as one package." -- Michael Nygard, Author of Release IT, Professional Programmer, and Software Architect "I've been reading the review copy of Dave Farley's book and it's what we need. It should be required reading for anyone aspiring to be a software engineer or who wants to master the craft. Pragmatic, practical advice on professional engineering. It should be required reading in universities and bootcamps." -- Bryan Finster, Distinguished Engineer and Value Stream Architect at USAF Platform One "The title says it all. In this book, Dave Farley shares his wisdom and experience as an outstanding software engineer and leader. The reader is fortunate to see the world of software design through the eyes of a master designer.  Modern - It describes the practices tools and technology used today to build working software with a productive cadence. Modern Software Engineering provides a reader with a clear understanding of the field of software engineering and why it is indeed engineering.  Dave explains the essential aspects of software engineering concisely from the perspective of a software engineer.  Unlike many books which focus on one right way, he stresses the importance of good judgment, experimentation, and measurement.  Many authors discuss the goodness of cohesion, coupling, and separation of concerns but Dave illustrates concepts while discussing the natural tensions between them which are part of the art of software design.  His passion and discussion for TDD and perspective on how and why it works, provide fresh motivation as TDD as design practice. Throughout the book, Dave presents concrete examples of design choices, where creative experimentation, measure, and iterative development are essential.  The book presents a series of courteous conversations about software product design and implementation.   It is a book that professionals will return to often to reread and think about these important design conversations. This is an excellent book that belongs on the self of every software engineer be they new or leading a large team."—Dave Thomas, CEO of Bedarra Corporation