Decision Analytics

Mathematical Models and Algorithms for Sequential Decision-Making

Inbunden, Engelska, 2027

1 919 kr

Kommande

Mathematical models and algorithms for sequential decisions under uncertainty Sequential decisions under uncertainty arise in many fields including energy, healthcare, finance, transportation, and logistics, yet accessible treatments linking foundational theory to computational practice remain scarce. Decision Analytics: Mathematical Models and Algorithms for Sequential Decision-Making, written by Brian T. Denton, a past President of INFORMS, presents a structured progression from core concepts through advanced methods, pairing rigorous mathematics with implementable Python code. Across ten chapters, Decision Analytics covers decision trees, Monte Carlo simulation, Markov chains, Markov decision processes, partially observable Markov decision processes, and constrained optimization models, including stochastic programs. Dedicated chapters on reinforcement learning and multi-agent learning introduce model-free approaches for finding optimal or near-optimal solutions. The final chapter covers approximate dynamic programming for decision-making at scale. Real-world examples, exercises, and an instructor's solution manual support classroom adoption. Readers will also find: Coverage of artificial intelligence techniques applied to sequential decision-making problemsMonte Carlo simulation methods used to analyse decision trees, Markov decision processes, and stochastic programming formulationsPython code examples throughout the text enabling direct implementation and experimentation with each model and algorithm presentedPractice exercises with solutions and an instructor's manual designed to support both self-study and classroom-based teachingA concept-first pedagogical approach that explains foundational principles before demonstrating how they solve applied problemsDesigned for undergraduate and graduate students in industrial engineering, operations research, and related STEM disciplines with introductory knowledge of mathematics, probability, and statistics, this book also serves researchers and professionals who require unified treatment of sequential decision-making methods.

Produktinformation

Utgivningsdatum2027-01-17
FormatInbunden
SpråkEngelska
Antal sidor256
FörlagJohn Wiley & Sons Inc
ISBN9781394345786

Tillhör följande kategorier

Optimering inom Naturvetenskap och teknik
Databaser inom Data och IT
Systemvetenskap och AI inom Data och IT

Brian T. Denton, PhD, is the Stephen M. Pollock Collegiate Professor of Industrial and Operations Engineering at the University of Michigan. A past President of INFORMS, his research focuses on data-driven sequential decision making and optimization under uncertainty. His work has earned the National Science Foundation Career Award, INFORMS Daniel H. Wagner Prize, and the George E. Kimball medal. He is a Fellow of INFORMS and the Institute of Industrial and Systems Engineers.

Table of ContentsAbout the authorAcknowledgement1 Introduction1.1 Introduction1.2 Mathematical Models for Decision Making1.3 Examples of Practical Applications1.4 Python and Computational Examples1.5 Summary of Future Chapters1.6 Concluding Remarks1.7 Practice Exercises2 Decision Trees2.1 Decision Trees2.2 Basic Probability Concepts2.3 Quantifying Decisions: Payoffs and Decision-Maker Perspectives2.3.1 Quality Adjusted Life Years2.3.2 Probability of an Event2.3.3 Utility2.3.4 Time Value of Rewards2.3.5 Regret2.4 Multi-stage Decision Trees2.5 Expected Value of Perfect Information2.6 Concluding Remarks2.7 Practice Exercises3 Deterministic Dynamic Programs3.1 Introduction3.2 Mathematical Formulation of Dynamic Programs3.3 Shortest Path Problems on Directed Acyclic Networks3.4 Production Lot-sizing3.5 Resource Allocation3.6 Pattern Recognition3.7 Generalization of Shortest Path Problems to Include Cycles3.8 Counter Example: When DP Does Not Work3.9 Concluding Remarks3.10 Practice Exercises4 Markov Decision Processes4.1 Introduction4.2 Markov Chains4.3 Markov Decision Processes (MDPs)4.3.1 Estimating Computational Complexity of Policy Evaluation4.3.2 Finding Optimal Policies Efficiently4.3.3 Analysis of the Backward Induction Algorithm4.3.4 Shortest Path Problem Revisited4.4 Optimal Stopping Time Problems4.5 Production Planning with Uncertain Demand4.6 MDP Parameter Estimation4.7 Infinite-Horizon MDPs4.7.1 Policy Evaluation Over an Infinite Horizon4.7.2 Optimality Equations for Infinite-Horizon MDPs4.7.3 Solving the Optimality Equations for Infinite-Horizon MDPsValue IterationThe Reasons Value Iteration WorksPolicy IterationThe Reason Policy Iteration Works4.8 Concluding Remarks4.9 Practice Exercises5 Constrained Optimization Models5.1 Introduction5.2 A Visual Introduction to Linear Programming Models5.3 The Shortest Path Problem Revisited5.4 Relationship Between Dynamic Programs and Linear Programs5.5 Two-Stage Stochastic Linear Programs5.5.1 The Newsvendor Problem5.5.2 Value of the Stochastic Solution5.5.3 Scheduling Arrivals to a Stochastic Server5.6 Expected Value of Perfect Information (EVPI)5.7 Concluding Remarks5.8 Practice Exercises6 Monte Carlo Simulation6.1 Introduction6.2 Curse of Dimensionality6.3 Monte Carlo Sampling6.3.1 Pseudo Random Number Generators6.4 Using Monte Carlo Simulation to Estimate Expectations6.5 Sampling a Markov Chain6.5.1 Using Monte Carlo Simulation to Analyze MDPs6.6 Approximating Stochastic Programs with Monte Carlo Simulation6.6.1 Monte Carlo Sampling for Multi-Stage Stochastic Programs6.7 Monte Carlo Sampling for MDPs6.8 Concluding Remarks6.9 Practice Exercises7 Partially Observable Markov Decision Processes (POMDPs)7.1 Introduction7.2 Hidden Markov Models (HMMs)7.3 HMM Parameter Estimation7.4 POMDP Model Formulation7.5 Solution Methods for POMDPs7.6 Approximation Methods for POMDPs7.7 Concluding Remarks7.8 Practice Exercises8 Reinforcement Learning8.1 Introduction8.2 Contexts and Types of Learning8.2.1 Off-line vs. On-line Learning8.2.2 Stationary vs. Non-stationary Environments8.2.3 Value Iteration vs. Policy Iteration Approaches8.2.4 On-policy vs. Off-policy Learning8.2.5 Exploitation vs. Exploration8.3 Greedy Monte Carlo Policy Iteration8.3.1 Greedy Policy for Bandit Problems8.3.2 ϵ-Greedy Approach to the Multi-Armed Bandit8.4 Monte-Carlo Policy Iteration8.5 Q-Learning8.6 Concluding Remarks8.7 Practice Exercises9 Multi-agent Markov Decision Processes9.1 Introduction9.2 A Two-Agent Model: Decision Maker vs. an Adversary9.3 Cooperative Multi-Agent Markov Decision Processes (MAMDPs)9.3.1 Independent Cooperative Agents with No Information Sharing9.3.2 Fully Centralized MAMDPs9.3.3 Fully Centralized MAMDPs with Partial Observability9.3.4 Decentralized POMDPs9.4 Concluding Remarks9.5 Practice Exercises10 Approximation Methods for Large-Scale Models10.1 Introduction10.2 Value Function Approximations10.2.1 Basis Function Approximations10.2.2 Kernel-Based Approximations10.3 Policy Iteration Based Approximation10.3.1 Policy Approximation Methods10.4 Concluding Remarks10.5 Practice ExercisesA Appendix A - Mathematical NotationA.1 Appendix B - Derivation of Newsvendor Model Optimal SolutionA.2 Appendix C - Theorem Proofs