Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes

Benoîte de Saporta is Professor in Applied Probabilities at the University of Montpellier 2 in France. François Dufour is Professor at the University of Bordeaux in France. Huilong Zhang is a lecturer at INRIA in Bordeaux, France.

• Preface ixIntroduction xiPart 1. Piecewise Deterministic Markov Processes and Quantization 1Chapter 1. Piecewise Deterministic Markov Processes 31.1. Introduction 31.2. Notation 41.3. Definition of a PDMP 51.4. Regularityassumptions 81.4.1. Lipschitz continuity along the flow 81.4.2. Regularity assumptions on the local characteristics 91.5. Time-augmentedprocess 111.6. EmbeddedMarkovchain 151.7. Stopping times 161.8. ExamplesofPDMPs 201.8.1. Poisson processwith trend 201.8.2. TCP 211.8.3. Air conditioningunit 221.8.4. Crack propagationmodel 231.8.5. Repairworkshopmodel 24Chapter 2. Examples in Reliability 272.1. Introduction 272.2. Structure subject to corrosion 282.2.1. PDMPmodel 292.2.2. Deterministic time to reach the boundary 322.3. The heatedhold-uptank 332.3.1. Tank dynamics 342.3.2. PDMPmodel 36Chapter 3. Quantization Technique 393.1. Introduction 393.2. Optimal quantization 403.2.1. Optimal quantization of a random variable 403.2.2. Optimal quantization of a Markovchain 423.3. SimulationofPDMPs 443.3.1. Simulation of time-dependent intensity 453.3.2. Simulation of trajectories 453.4. QuantizationofPDMPs 473.4.1. Scale of coordinates of the state variable 483.4.2. Cardinality of the mode variable 50Part 2. Simulation of Functionals 53Chapter 4. Expectation of Functionals 554.1. Introduction 554.2. Recursive formulation 574.2.1. Lipschitz continuity 584.2.2. Iterated operator 604.2.3. Approximationscheme 614.3. Lipschitz regularity 624.4. Rate of convergence 694.5. Time-dependent functionals 714.6. Deterministic time horizon 744.6.1. Direct estimation of the running cost term 744.6.2. Bounds of the boundary jump cost term 774.6.3. Bounds in the general case 794.7. Example 814.8. Conclusion 84Chapter 5. Exit Time 875.1. Introduction 875.2. Problem setting 885.2.1. Distribution 905.2.2. Moments 915.2.3. Computationhorizon 925.3. Approximationschemes 925.4. Convergence 955.4.1. Distribution 955.4.2. Moments 1005.5. Example 1015.6. Conclusion 108Chapter 6. Example in Reliability: Service Time 1096.1. Mean thickness loss 1096.2. Service time 1126.2.1. Mean service time 1146.2.2. Distribution of the service time 1186.3. Conclusion 121Part 3. Optimization 123Chapter 7. Optimal Stopping 1257.1. Introduction 1257.2. Dynamic programming equation 1287.3. Approximation of the value function 1307.4. Lipschitz continuity properties 1327.4.1. Lipschitz properties of J and K 1327.4.2. Lipschitz properties of the value functions 1357.5. Error estimation for the value function 1387.5.1. Second term 1407.5.2. Third term 1417.5.3. Fourth term 1477.5.4. Proof of theorem 7.1 1487.6. Numerical construction of an [1]-optimal stopping time 1497.7. Example 161Chapter 8. Partially Observed Optimal Stopping Problem 1658.1. Introduction 1658.2. Problem formulation and assumptions 1678.3. Optimal filtering 1708.4. Dynamicprogramming 1758.4.1. Preliminaryresults 1768.4.2. Optimal stopping problem under complete observation 1808.4.3. Dynamic programming equation 1818.5. Numerical approximation by quantization 1888.5.1. Lipschitz properties 1898.5.2. Discretization scheme 1958.5.3. Numerical construction of an [1]-optimal stopping time 2058.6. Numerical example 211Chapter 9. Example in Reliability: Maintenance Optimization 2159.1. Introduction 2159.2. Corrosionprocess 2169.3. Air conditioningunit 2199.4. The heatedhold-uptank 2219.4.1. Problem setting and simulation 2229.4.2. Numerical results and validation 2249.5. Conclusion 228Chapter 10. Optimal Impulse Control 23110.1. Introduction 23110.2. Impulse controlproblem23310.3. Lipschitz-continuity properties 23610.3.1. Lipschitz properties of the operators 23610.3.2. Lipschitz properties of the operator L 23910.4. Approximation of the value function 24210.4.1. Time discretization 24510.4.2. Approximation of the value functions on the control grid U 24610.4.3. Approximation of the value function 25510.4.4. Step-by-step description of the algorithm 25910.4.5. Practical implementation 25910.5. Example 26210.6. Conclusion 264Bibliography 269Index 277