Continuous-time Markov chains and applications

by George Yin

This book discusses continuous-time Markov chains and applications. Using a singular perturbation approach, it presents a systematic treatment of singularly perturbed systems that naturally arise in queueing theory, control and optimization, and manufacturing systems. It gathers a number of ideas in Markov chains and singular perturbations that are scattered throughout the literature. It presents results on asymptotic expansions of the corresponding probability distributions, functional occupation measures, exponential upper bounds, and asymptotic normality. The emphasis is on Markov chains with weak and strong interactions and structural properties. Much of the content is an outgrowth of the authors' recent research. Some of the results have not appeared elsewhere. This book will be an important reference for researchers in applied mathematics, probability and stochastic processes, operations research, control theory, and optimization.