Introduction to Stochastic Models

Marius Iosifescu is a member and vice-president of the Romanian Academy of Sciences and director of the Bucharest Institute of Mathematical Statistics and Applied Mathematics. His research concerns stochastic processes, the probabilistic theory of numbers and their applications.Nikolaos Limnios is a professor of applied mathematics at the University of Technology of Compiègne, France. His research and teaching activities concern stochastic processes, statistical inference and their applications.Gheorghe Oprisan is a professor of applied mathematics at the 'Politehnica' University of Bucharest and the head of the Mathematical section II there. His research concerns stochastic processes and their applications.

• Preface ixChapter 1. Introduction to Stochastic Processes 11.1. Sequences of random variables 11.2. The notion of stochastic process 101.3. Martingales 131.4. Markov chains 171.5. State classification 241.6. Continuous-time Markov processes 271.7. Semi-Markov processes 33Chapter 2. Simple Stochastic Models 372.1. Urn models 372.2. Random walks 392.3. Brownian motion 442.4. Poisson processes 502.5. Birth and death processes 59Chapter 3. Elements of Markov Modeling 613.1. Markov models: ideas, history, applications 613.2. The discrete-time Ehrenfest model 633.3. Markov models in genetics 793.4. Markov storage models 1103.5. Reliability of Markov models 124Chapter 4. Renewal Models 1494.1. Fundamental concepts and examples 1494.2. Waiting times 1554.3. Modified renewal processes 1594.4. Replacement models 1614.5. Renewal reward processes 1654.6. The risk problem of an insurance company 1684.7. Counter models 1714.8. Alternating renewal processes 1804.9. Superposition of renewal processes 1824.10. Regenerative processes 186Chapter 5. Semi-Markov Models 1895.1. Introduction 1895.2. Markov renewal processes 1905.3. First-passage times and state classification 1965.4. Reliability 2005.5. Reservoir models 2075.6. Queues 2185.7. Digital communication channels 222Chapter 6. Branching Models 2276.1. The Bienaymé-Galton-Watson model 2276.2. Generalizations of the B-G-W model 2716.3. Continuous-time models 302Chapter 7. Optimal Stopping Models 3157.1. The classic optimal stopping problem 3157.2. Renewal with binary decision 333Bibliography 343Notation 367Index 369