You are here

Markov Chains and Stochastic Stability

Sean Meyn and Richard L. Tweedie
Cambridge University Press
Publication Date: 
Number of Pages: 
[Reviewed by
Manan Shah
, on

Meyn and Tweedie’s 2nd edition of Markov Chains and Stochastic Stability is a very well-written, well-organized text, suitable for practitioners and ambitious graduate students from a variety of fields such as probability, operations research, mathematical finance, and computer science. The notation is clear and consistent. Definitions and theorems are easy to find and introduced when necessary. The chapter introductions give a clear, high-level view of the topics to be discussed, thus the uninitiated can have an intuitive basis to help them understand the material.

There are no problems or exercises. Thus, it could not be used as a stand-alone course text. However, a lecturer can very well use the numerous examples provided as a real guide for explaining, in practical terms, the key concepts of stability for Markov chains.

The authors take great care to use terminology that is familiar to folks from a multitude of research areas. As an example, in Chapter 2, the iid random variables W = {Wn} which are often used as a random “shock” in a model, go by many names. The authors make the reader aware that W, depending on the context, is called error, noise, disturbance, innovation, and increments, to name a few. This is an advantage since sometimes the barrier to entry into a new field is the language rather than the underlying mathematics.

This book is divided into three parts. Part 1 (Chapters 1 to 7), titled Communication and Regeneration, introduces the basics (Chapters 1 to 3) and then focuses on the notion of (ψ-) irreducibility of chains. Part 2 (Chapters 8 to 12), titled Stability Structures, builds on the study of ψ-irreducibility to develop the idea of transience and recurrence. Finally, Part 3 (Chapters 13 to 20), titled Convergence, focuses entirely on ergodicity.

The authors assume that the reader has had some experience with Markov chains on countable spaces. The appendices, while terse, provide a sufficient reference. The appendices’ topics are as follows: Appendix A discusses transience vs. recurrence, positivity vs. nullity, and convergence, Appendix B gives a glossary of drift conditions and an analysis of the SETAR (scalar threshold autoregression) model, Appendix C reviews the assumptions of the models used throughout the text, and Appendix D provides reference for the mathematical background assumed in the text, mainly, measure theory in the context of probability theory, a broad overview of martingale theory, and some topology.

Practical and common Markovian models are seamlessly introduced and developed with the theory. It is these models that tie the three parts of this book together from an “applied” standpoint. For example, the random walk (on R or R+) is used in Section 3.3 to illustrate how one generates transition matrices, Section 3.5 (Building transition kernels for specific models), Section 4.1 to motivate the notions of communication and irreducibility, Section 4.3 for ψ-irreducibility, Chapter 8 on transience and recurrence and the classification of ψ-irreducible chains, and later again in Chapters 15 and 16 when discussing different types of ergodicity. Thus, a reader can readily see through theory and consistent examples, how ψ-irreducibility, recurrence, and ergodicity are linked. Queueing models are used similarly appear throughout the text as well.

The mixture of theory with well-explained examples makes Meyn and Tweedie’s text an excellent resource for both practitioners and theoreticians.

Manan Shah is adjunct faculty at Ramapo College of New Jersey. He teaches Probability & Statistics in the School of Theoretical and Applied Sciences. He is reachable at


List of figures; Prologue to the second edition Peter W. Glynn; Preface to the second edition Sean Meyn; Preface to the first edition; Part I. Communication and Regeneration: 1. Heuristics; 2. Markov models; 3. Transition probabilities; 4. Irreducibility; 5. Pseudo-atoms; 6. Topology and continuity; 7. The nonlinear state space model; Part II. Stability Structures: 8. Transience and recurrence; 9. Harris and topological recurrence; 10. The existence of Π; 11. Drift and regularity; 12. Invariance and tightness; Part III. Convergence: 13. Ergodicity; 14. f-Ergodicity and f-regularity; 15. Geometric ergodicity; 16. V-Uniform ergodicity; 17. Sample paths and limit theorems; 18. Positivity; 19. Generalized classification criteria; 20. Epilogue to the second edition; Part IV. Appendices: A. Mud maps; B. Testing for stability; C. Glossary of model assumptions; D. Some mathematical background; Bibliography; Indexes.