Kristen Kehrer has been providing innovative and practical statistical modeling solutions since 2010. In 2018, she achieved recognition as a LinkedIn Top Voice in Data Science & Analytics. Kristen is also the founder of Data Moves Me, LLC. Caleb Kaiser is a Full Stack Engineer at Comet. Caleb was previously on the Founding Team at Cortex Labs. Caleb also worked at Scribe Media on the Author Platform Team.

• Introduction ix1 A Gentle Introduction to Modern Machine Learning 1Data Science Is Diverging from Business Intelligence 3From CRISP-DM to Modern, Multicomponent ml Systems 4The Emergence of LLMs Has Increased ML’s Power and Complexity 7What You Can Expect from This Book 92 An End-to-End Approach 11Components of a YouTube Search Agent 13Principles of a Production Machine Learning System 16Observability 19Reproducibility 19Interoperability 20Scalability 21Improvability 22A Note on Tools 233 A Data-Centric View 25The Emergence of Foundation Models 25The Role of Off-the-Shelf Components 27The Data-Driven Approach 28A Note on Data Ethics 28Building the Dataset 30Working with Vector Databases 34Data Versioning and Management 50Getting Started with Data Versioning 53Knowing “Just Enough” Engineering 574 Standing Up Your LLM 61Selecting Your LLM 61What Type of Inference Do I Need to Perform? 65How Open-Ended Is This Task? 66What Are the Privacy Concerns for This Data? 66How Much Will This Model Cost? 67Experiment Management with LLMs 68LLM Inference 74Basics of Prompt Engineering 74In-Context Learning 77Intermediary Computation 85Augmented Generation 89Agentic Techniques 94Optimizing LLM Inference with Experiment Management 102Fine-Tuning LLMs 111When to Fine-Tune an LLM 112Quantization, QLOrA, and Parameter Efficient Fine-Tuning 113Wrapping Things Up 1215 Putting Together an Application 123Prototyping with Gradio 125Creating Graphics with Plotnine 128Adding the Author Selector 137Adding a Logo 138Adding a Tab 139Adding a Title and Subtitle 140Changing the Color of the Buttons 140Click to Download Button 141Putting It All Together 141Deploying Models as APIs 144Implementing an API with FastAPI 146Implementing Uvicorn 148Monitoring an LLM 149Dockerizing Your Service 151Deploying Your Own LLM 154Wrapping Things Up 1596 Rounding Out the ML Life Cycle 161Deploying a Simple Random Forest Model 161An Introduction to Model Monitoring 167Model Monitoring with Evidently AI 175Building a Model Monitoring System 176Final Thoughts on Monitoring 1877 Review of Best Practices 189Step 1: Understand the Problem 189Step 2: Model Selection and Training 190Step 3: Deploy and Maintain 192Step 4: Collaborate and Communicate 196Emerging Trends in LLMs 197Next Steps in Learning 199Appendix: Additional LLM Example 201Index 209

Kristen Kehrer has been providing innovative and practical statistical modeling solutions since 2010. In 2018, she achieved recognition as a LinkedIn Top Voice in Data Science & Analytics. Kristen is also the founder of Data Moves Me, LLC. Caleb Kaiser is a Full Stack Engineer at Comet. Caleb was previously on the Founding Team at Cortex Labs. Caleb also worked at Scribe Media on the Author Platform Team.

• Introduction ix1 A Gentle Introduction to Modern Machine Learning 1Data Science Is Diverging from Business Intelligence 3From CRISP-DM to Modern, Multicomponent ml Systems 4The Emergence of LLMs Has Increased ML’s Power and Complexity 7What You Can Expect from This Book 92 An End-to-End Approach 11Components of a YouTube Search Agent 13Principles of a Production Machine Learning System 16Observability 19Reproducibility 19Interoperability 20Scalability 21Improvability 22A Note on Tools 233 A Data-Centric View 25The Emergence of Foundation Models 25The Role of Off-the-Shelf Components 27The Data-Driven Approach 28A Note on Data Ethics 28Building the Dataset 30Working with Vector Databases 34Data Versioning and Management 50Getting Started with Data Versioning 53Knowing “Just Enough” Engineering 574 Standing Up Your LLM 61Selecting Your LLM 61What Type of Inference Do I Need to Perform? 65How Open-Ended Is This Task? 66What Are the Privacy Concerns for This Data? 66How Much Will This Model Cost? 67Experiment Management with LLMs 68LLM Inference 74Basics of Prompt Engineering 74In-Context Learning 77Intermediary Computation 85Augmented Generation 89Agentic Techniques 94Optimizing LLM Inference with Experiment Management 102Fine-Tuning LLMs 111When to Fine-Tune an LLM 112Quantization, QLOrA, and Parameter Efficient Fine-Tuning 113Wrapping Things Up 1215 Putting Together an Application 123Prototyping with Gradio 125Creating Graphics with Plotnine 128Adding the Author Selector 137Adding a Logo 138Adding a Tab 139Adding a Title and Subtitle 140Changing the Color of the Buttons 140Click to Download Button 141Putting It All Together 141Deploying Models as APIs 144Implementing an API with FastAPI 146Implementing Uvicorn 148Monitoring an LLM 149Dockerizing Your Service 151Deploying Your Own LLM 154Wrapping Things Up 1596 Rounding Out the ML Life Cycle 161Deploying a Simple Random Forest Model 161An Introduction to Model Monitoring 167Model Monitoring with Evidently AI 175Building a Model Monitoring System 176Final Thoughts on Monitoring 1877 Review of Best Practices 189Step 1: Understand the Problem 189Step 2: Model Selection and Training 190Step 3: Deploy and Maintain 192Step 4: Collaborate and Communicate 196Emerging Trends in LLMs 197Next Steps in Learning 199Appendix: Additional LLM Example 201Index 209

Kristen Kehrer has been providing innovative and practical statistical modeling solutions since 2010. In 2018, she achieved recognition as a LinkedIn Top Voice in Data Science & Analytics. Kristen is also the founder of Data Moves Me, LLC. Caleb Kaiser is a Full Stack Engineer at Comet. Caleb was previously on the Founding Team at Cortex Labs. Caleb also worked at Scribe Media on the Author Platform Team.

• Introduction ix1 A Gentle Introduction to Modern Machine Learning 1Data Science Is Diverging from Business Intelligence 3From CRISP-DM to Modern, Multicomponent ml Systems 4The Emergence of LLMs Has Increased ML’s Power and Complexity 7What You Can Expect from This Book 92 An End-to-End Approach 11Components of a YouTube Search Agent 13Principles of a Production Machine Learning System 16Observability 19Reproducibility 19Interoperability 20Scalability 21Improvability 22A Note on Tools 233 A Data-Centric View 25The Emergence of Foundation Models 25The Role of Off-the-Shelf Components 27The Data-Driven Approach 28A Note on Data Ethics 28Building the Dataset 30Working with Vector Databases 34Data Versioning and Management 50Getting Started with Data Versioning 53Knowing “Just Enough” Engineering 574 Standing Up Your LLM 61Selecting Your LLM 61What Type of Inference Do I Need to Perform? 65How Open-Ended Is This Task? 66What Are the Privacy Concerns for This Data? 66How Much Will This Model Cost? 67Experiment Management with LLMs 68LLM Inference 74Basics of Prompt Engineering 74In-Context Learning 77Intermediary Computation 85Augmented Generation 89Agentic Techniques 94Optimizing LLM Inference with Experiment Management 102Fine-Tuning LLMs 111When to Fine-Tune an LLM 112Quantization, QLOrA, and Parameter Efficient Fine-Tuning 113Wrapping Things Up 1215 Putting Together an Application 123Prototyping with Gradio 125Creating Graphics with Plotnine 128Adding the Author Selector 137Adding a Logo 138Adding a Tab 139Adding a Title and Subtitle 140Changing the Color of the Buttons 140Click to Download Button 141Putting It All Together 141Deploying Models as APIs 144Implementing an API with FastAPI 146Implementing Uvicorn 148Monitoring an LLM 149Dockerizing Your Service 151Deploying Your Own LLM 154Wrapping Things Up 1596 Rounding Out the ML Life Cycle 161Deploying a Simple Random Forest Model 161An Introduction to Model Monitoring 167Model Monitoring with Evidently AI 175Building a Model Monitoring System 176Final Thoughts on Monitoring 1877 Review of Best Practices 189Step 1: Understand the Problem 189Step 2: Model Selection and Training 190Step 3: Deploy and Maintain 192Step 4: Collaborate and Communicate 196Emerging Trends in LLMs 197Next Steps in Learning 199Appendix: Additional LLM Example 201Index 209