Modeling and Analysis of Stochastic Systems: (Chapman & Hall/CRC Texts in Statistical Science)

Building on the author’s more than 35 years of teaching experience, Modeling and Analysis of Stochastic Systems, Third Edition, covers the most important classes of stochastic processes used in the modeling of diverse systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models. The third edition has been updated with several new applications, including the Google search algorithm in discrete time Markov chains, several examples from health care and finance in continuous time Markov chains, and square root staffing rule in Queuing models. More than 50 new exercises have been added to enhance its use as a course text or for self-study. The sequence of chapters and exercises has been maintained between editions, to enable those now teaching from the second edition to use the third edition. Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, readers will be well-equipped to build and analyze useful stochastic models for real-life situations.

Table of Contents:
IntroductionWhat in the World is a Stochastic Process? How to Characterize a Stochastic Process What Do We Do with a Stochastic Process? Discrete-Time Markov Chains: Transient Behaviour Definition and Characterization Examples DTMCs in Other Fields Marginal Distributions Occupancy Times Computation of Matrix Powers Modeling Exercises Computational Exercises Conceptual Exercises Discrete-Time Markov Chains: First Passage Times Definitions Cumulative Distribution Function of T Absorption Probabilities Expectation of T Generating Function and Higher Moments of T Computational Exercises Conceptual Exercises Discrete-Time Markox Chains: Limiting Behaviour Exploring the Limiting Behaviour by Examples Classification of States Determining Recurrence and Transience: Finite DTMCs Determining Recurrence and Transience: Infinite DTMSc Limiting Behaviour of Irreducible DTMCs Examples: Limiting Behaviour of Infinite State-Space Irreducible DTMCs Limiting Behaviour of Reducible DTMCs DTMCs with Costs and Rewards Reversibility Computational Exercises Conceptual Exercises Poisson Processes Exponential Distributions Poisson Process: Definitions Event Times in a Poisson Process Superposition and Splitting of Poisson Processes Non-Homogeneous Poisson Process Compound Poisson Process Computational Exercises Conceptual Exercises Continuous-Time Markov Chains Definitions and Sample Path Properties Examples CTMCs in Other Fields Transient Behaviour: Marginal Distribution Transient Behaviour: Occupancy Times Computation of P(t): Finite State-Space Computation of P(t): Infinite State-Space First-Passage Times Exploring the Limiting Behaviour by Examples Classification of States Limiting Behaviour of Irreducible CTMCs Limiting Behaviour of Reducible CTMCs CTMCs with Costs and Rewards Phase Type Distributions Reversibility Modeling Exercises Computational Exercises Conceptual Exercises Queueing Models Introduction Properties of General Queueing Systems Birth and Death Queues Open Queueing Networks Closed Queueing Networks Single Server Queues Retrial Queue Infinite Server Queue Modeling Exercises Computational Exercises Renewal Processes Introduction Properties of N(t) The Renewal Function Renewal-Type Equation Key Renewal Theorem Recurrence Times Delayed Renewal Processes Semi-Markov Processes Renewal Processes with Costs/Rewards Regenerative Processes Computational Exercises Conceptual Exercises Markov Regenerative Processes Definitions and Examples Markov Renewal Process and Markov Renewal Function Key Renewal Theorem for MRPs Semi-Markov Processes: Further Results Markov Regenerative Processes Applications to Queues Modeling Exercises Computational Exercises Conceptual Exercises Diffusion Process Brownian Motion Sample Path Properties of BM Kolmogorov Equations for Standard Brownian Motion First Passage Times Reflected SBM Reflected BM and Limiting Distributions BM and Martingales Cost/Reward Models Stochastic Integration Stochastic Differential Equations and Ito's Formula Applications to Finance Computational Exercises Conceptual Exercises Epilogue Probability of Events AppendicesUnivariate Random Variables Multivariate Random Variables Generating Functions Laplace-Stieltjes Transforms Laplace Transforms Modes of Convergence Results from Analysis Difference and Differential Equations Answers to Selected Problems References Index