Continuous-Time Markov Chains and Applications

• ​ Prologue and Preliminaries: Introduction and overview- Mathematical preliminaries.- Markovian models.- Two-Time-Scale Markov Chains: Asymptotic Expansions of Solutions for Forward Equations.- Occupation Measures: Asymptotic Properties and Ramification.- Asymptotic Expansions of Solutions for Backward Equations.- Applications:MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching: Markov Decision Problems.- Stochastic Control of Dynamical Systems.- Numerical Methods for Control and Optimization.- Hybrid LQG Problems.- References.- Index.-

From the reviews of the second edition: "This book is the expanded second edition of 'Continuous-time Markov chains and applications. A singular perturbation approach.' which appeared 1998. ... The book remains clearly of interest to researchers in stochastic control, operation research, manufacturing system, engineering, economics and applied mathematics." (Michael Hogele, zbMATH, Vol. 1277, 2014) "The book is devoted to a study of continuous time singularly perturbed (SP) Markov chains and their applications in problems of control and optimization. ... There is no doubt that this second, revised and updated edition will be also well received by the intended audience (researchers and graduate students in applied mathematics and control engineering interested in modeling and optimization of complex stochastic systems) and will excite further interest to the topic." (Vladimir Gaitsgory, SIAM Review, Vol. 55 (4), 2013)