Engineering AI Systems

Dr. Len Bass is a seasoned researcher with over 30 years in software architecture and more than a decade in DevOps. He has been teaching DevOps to graduate students for seven years and is the author of a bestselling book on software architecture, along with three books on DevOps. Dr. Qinghua Lu is a principal research scientist at CSIRO's Data61, leading the Software Engineering for AI and Responsible AI science teams. She is a coauthor of Responsible AI: Best Practices for Creating Trustworthy AI Systems (Addison-Wesley, 2024). Prof. Dr. Ingo Weber is a professor at the Technical University of Munich and Director of Digital Transformation and ICT Infrastructure at Fraunhofer-Gesellschaft. He has written numerous publications and textbooks, including DevOps: A Software Architect’s Perspective and Architecture for Blockchain Applications. Dr. Liming Zhu is a research director at CSIRO's Data61 and is a conjoint professor at University of New South Wales. He contributes to various AI safety and standards committees and has written over 300 papers. He is coauthor of Responsible AI: Best Practices for Creating Trustworthy AI Systems.

• Preface xiiiAcknowledgments xviiAbout the Authors xixChapter 1: Introduction 11.1 What We Talk about When We Talk about Things: Terminology 21.2 Achieving System Qualities 41.3 Life-Cycle Processes 61.4 Software Architecture 101.5 AI Model Quality 131.6 Dealing with Uncertainty 191.7 Summary 201.8 Discussion Questions 211.9 For Further Reading 21Chapter 2: Software Engineering Background 232.1 Distributed Computing 232.2 DevOps Background 352.3 MLOps Background 422.4 Summary 442.5 Discussion Questions 452.6 For Further Reading 45Chapter 3: AI Background 473.1 Terminology 483.2 Selecting a Model 493.3 Preparing the Model for Training 653.4 Summary 693.5 Discussion Questions 693.6 For Further Reading 69Chapter 4: Foundation Models 714.1 Foundation Models 714.2 Transformer Architecture 724.3 Alternatives in FM Architectures 744.4 Customizing FMs 754.5 Designing a System Using FMs 864.6 Maturity of FMs and Organizations 914.7 Challenges of FMs 934.8 Summary 944.9 Discussion Questions 944.10 For Further Reading 94Chapter 5: AI Model Life Cycle 975.1 Developing the Model 975.2 Building the Model 1085.3 Testing the Model 1095.4 Release 1145.5 Summary 1145.6 Discussion Questions 1155.7 For Further Reading 115Chapter 6: System Life Cycle 1176.1 Design 1186.2 Developing Non-AI Modules 1216.3 Build 1226.4 Test 1236.5 Release and Deploy 1256.6 Operate, Monitor, and Analyze 1356.7 Summary 1406.8 Discussion Questions 1416.9 For Further Reading 141Chapter 7: Reliability 1437.1 Fundamental Concepts 1437.2 Preventing Faults 1457.3 Detecting Faults 1497.4 Recovering from Faults 1527.5 Summary 1547.6 Discussion Questions 1547.7 For Further Reading 154Chapter 8: Performance 1558.1 Efficiency 1558.2 Accuracy 1648.3 Summary 1738.4 Discussion Questions 1738.5 For Further Reading 174Chapter 9: Security 1759.1 Fundamental Concepts 1769.2 Approaches to Mitigating Security Concerns 1809.3 Summary 1889.4 Discussion Questions 1899.5 For Further Reading 189Chapter 10: Privacy and Fairness 19110.1 Privacy in AI Systems 19210.2 Fairness in AI Systems 19310.3 Achieving Privacy 19410.4 Achieving Fairness 19710.5 Summary 20110.6 Discussion Questions 20110.7 For Further Reading 202Chapter 11: Observability 20311.1 Fundamental Concepts 20311.2 Evolving from Monitorability to Observability 20411.3 Approaches for Enhancing Observability 20711.4 Summary 21111.5 Discussion Questions 21111.6 For Further Reading 212Chapter 12: The Fraunhofer Case Study: Using a Pretrained Language Model for Tendering 21312.1 The Problem Context 21412.2 Case Study Description and Setup 21712.3 Summary 23212.4 Takeaways 23312.5 Discussion Questions 23312.6 For Further Reading 233Chapter 13: The ARM Hub Case Study: Chatbots for Small and Medium-Size Australian Enterprises 23513.1 Introduction 23513.2 Our Approach 23613.3 LLMs in SME Manufacturing 23813.4 A RAG-Based Chatbot for SME Manufacturing 23813.5 Architecture of the ARM Hub Chatbot 23913.6 MLOps in ARM Hub 24413.7 Ongoing Work 25113.8 Summary 25213.9 Takeaways 25313.10 Discussion Questions 25413.11 For Further Reading 254Chapter 14: The Banking Case Study: Predicting Customer Churn in Banks 25514.1 Customer Churn Prediction 25614.2 Key Challenges in the Banking Sector 26514.3 Summary 26514.4 Takeaways 26614.5 Discussion Questions 26614.6 For Further Reading 267Chapter 15: The Future of AI Engineering 26915.1 The Shift to DevOps 2.0 27015.2 AI's Implications for the Future 27115.3 AIWare or AI-as-Software 27615.4 Trust in AI and the Role of Human Engineers 27915.5 Summary 28015.6 Discussion Questions 28115.7 For Further Reading 281References 283Index 289