Introduction to Stochastic Models

Marius Iosifescu is the author of Introduction to Stochastic Models, published by Wiley.Nikolaos Limnios is a Professor at the University of Technology, Compiègne, France. His research includes reliability, applied stochastic processes and statistics. He has written and edited many books in the reliability field.Gheorghe Oprisan is the author of Introduction to Stochastic Models, published by Wiley.

• Preface ixChapter 1. Introduction to Stochastic Processes 11.1. Sequences of random variables 11.2. The notion of stochastic process 101.3. Martingales 131.4. Markov chains 171.5. State classification 241.6. Continuous-time Markov processes 271.7. Semi-Markov processes 33Chapter 2. Simple Stochastic Models 372.1. Urn models 372.2. Random walks 392.3. Brownian motion 442.4. Poisson processes 502.5. Birth and death processes 59Chapter 3. Elements of Markov Modeling 613.1. Markov models: ideas, history, applications 613.2. The discrete-time Ehrenfest model 633.3. Markov models in genetics 793.4. Markov storage models 1103.5. Reliability of Markov models 124Chapter 4. Renewal Models 1494.1. Fundamental concepts and examples 1494.2. Waiting times 1554.3. Modified renewal processes 1594.4. Replacement models 1614.5. Renewal reward processes 1654.6. The risk problem of an insurance company 1684.7. Counter models 1714.8. Alternating renewal processes 1804.9. Superposition of renewal processes 1824.10. Regenerative processes 186Chapter 5. Semi-Markov Models 1895.1. Introduction 1895.2. Markov renewal processes 1905.3. First-passage times and state classification 1965.4. Reliability 2005.5. Reservoir models 2075.6. Queues 2185.7. Digital communication channels 222Chapter 6. Branching Models 2276.1. The Bienaymé-Galton-Watson model 2276.2. Generalizations of the B-G-W model 2716.3. Continuous-time models 302Chapter 7. Optimal Stopping Models 3157.1. The classic optimal stopping problem 3157.2. Renewal with binary decision 333Bibliography 343Notation 367Index 369