Fundamentals of Stochastic Models

Professor Zhe George Zhang is a professor of Management Science in the Department of Decision Sciences, at the College of Business and Economics, Western Washington University. He has published more than 110 papers in prestigious journals such as Management Science, Operations Research, Manufacturing & Service Operations Management, Production and Operations Management, IIE Transactions, IEEE Transactions, Queueing Systems, Journal of Applied Probability. Co-authored with N. Tian, he published the research monograph entitled Vacation Queueing Models – Theory and Applications in 2006 by Springer, which is the first book on this topic and has been widely cited since its publication. Professor Zhang is one of the Editors-in-Chief for Journal of the Operational Research Society, the first Operational Research journal in the world, and one of the founding Editors-in-Chief for new journal Queueing Models and Service Management and is on the editorial board of several international journals.

• 1. Introduction. Part I. Fundamentals of Stochastic Models. 2. Discrete-time Markov Chains. 3. Continuous-Time Markov Chains. 4. Structured Markov Chains. 5. Renewal Processes and Embedded Markov Chains. 6. Random Walks and Brownian Motions. 7. Reflected Brownian Motion Approximations to Simple Stochastic Systems. 8. Large Queueing Systems. 9. Static Optimization in Stochastic Models. 10. Dynamic Optimization in Stochastic Models. 11. Learning in Stochastic Models. Part II. Appendices: Elements of Probability and Stochastics. A. Basics of Probability Theory. B. Conditional Expectation and Martingales. C. Some Useful Bounds, Inequalities, and Limit Laws. D. Non-linear Programming in Stochastics. E. Change of Probability Measure for a Normal Random Variable. F. Convergence of Random Variables. G. Major Theorems for Stochastic Process Limits. H. A Brief Review on Stochastic Calculus. I. Comparison of Stochastic Processes - Stochastic Orders. J. Matrix Algebra and Markov Chains.