Sean Meyn is a professor in the Department of Electrical and Computer Engineering and director of the Division and Control Laboratory of the Coordinated Science Laboratory at the University of Illinois. He has served on the editorial boards of several journals in areas of systems and control and applied probability. Richard L. Tweedie was Professor and Head of the Division of Biostatistics at the University of Minnesota before his death in 2001.

• List of figures; Prologue to the second edition Peter W. Glynn; Preface to the second edition Sean Meyn; Preface to the first edition; Part I. Communication and Regeneration: 1. Heuristics; 2. Markov models; 3. Transition probabilities; 4. Irreducibility; 5. Pseudo-atoms; 6. Topology and continuity; 7. The nonlinear state space model; Part II. Stability Structures: 8. Transience and recurrence; 9. Harris and topological recurrence; 10. The existence of Π; 11. Drift and regularity; 12. Invariance and tightness; Part III. Convergence: 13. Ergodicity; 14. f-Ergodicity and f-regularity; 15. Geometric ergodicity; 16. V-Uniform ergodicity; 17. Sample paths and limit theorems; 18. Positivity; 19. Generalized classification criteria; 20. Epilogue to the second edition; Part IV. Appendices: A. Mud maps; B. Testing for stability; C. Glossary of model assumptions; D. Some mathematical background; Bibliography; Indexes.

'This second edition remains true to the remarkable standards of scholarship established by the first edition … it will no doubt be a very welcome addition to the literature.' Peter W. Glynn, Prologue to the Second Edition