Modeling and Analysis of Stochastic Systems, Second Edition

Based on the author's more than 25 years of teaching experience, Modeling and Analysis of Stochastic Systems, Second Edition covers the most important classes of stochastic processes used in the modeling of diverse systems, from supply chains and inventory systems to genetics and biological systems. For each class of stochastic process, the text includes its definition, characterization, applications, transient and limiting behavior, first passage times, and cost/reward models. Along with reorganizing the material, this edition revises and adds new exercises and examples. New to the second edition: a new chapter on diffusion processes that gives an accessible and non-measure-theoretic treatment with applications to finance; a more streamlined, application-oriented approach to renewal, regenerative, and Markov regenerative processes; and, two appendices that collect relevant results from analysis and differential and difference equations. Rather than offer special tricks that work in specific problems, this book provides thorough coverage of general tools that enable the solution and analysis of stochastic models. After mastering the material in the text, students will be well-equipped to build and analyze useful stochastic models for various situations. A collection of MATLAB[registered]-based programs can be downloaded from the author's website and a solutions manual is available for qualifying instructors.

Table of Contents:
Introduction What in the World Is a Stochastic Process? How to Characterize a Stochastic Process What Do We Do with a Stochastic Process? Discrete-Time Markov Chains: Transient Behavior Definition and Characterization Examples DTMCs in Other Fields Marginal Distributions Occupancy Times Computation of Matrix Powers DTMCs: First Passage Times Definitions Cumulative Distribution Function of T Absorption Probabilities Expectation of T Generating Function and Higher Moments of T DTMCs: Limiting Behavior Exploring the Limiting Behavior by Examples Irreducibility and Periodicity Recurrence and Transience Determining Recurrence and Transience: Infinite DTMCs Limiting Behavior of Irreducible DTMCs Examples: Limiting Behavior of Infinite State-Space Irreducible DTMCs Limiting Behavior of Reducible DTMCs DTMCs with Costs and Rewards Reversibility Poisson Processes Exponential Distributions Poisson Process: Definitions Event Times in a Poisson Process Superposition and Splitting of Poisson Processes Non-Homogenous Poisson Process Compound Poisson Process Continuous-Time Markov Chains Definitions and Sample Path Properties Examples Transient Behavior: Marginal Distribution Transient Behavior: Occupancy Times Computation of P(t): Finite State-Space Computation of P(t): Infinite State-Space First-Passage Times Exploring the Limiting Behavior by Examples Classification of States Limiting Behavior of Irreducible CTMCs Limiting Behavior of Reducible CTMCs CTMCs with Costs and Rewards Phase-Type Distributions Reversibility Queueing Models Introduction Properties of General Queueing Systems Birth and Death Queues Open Queueing Networks Closed Queueing Networks Single Server Queues Retrial Queue Infinite Server Queue Renewal Processes Introduction Properties of N(t) The Renewal Function Renewal-Type Equation Key Renewal Theorem Recurrence Times Delayed Renewal Processes Alternating Renewal Processes Semi-Markov Processes Renewal Processes with Costs/Rewards Regenerative Processes Markov Regenerative Processes Definitions and Examples Markov Renewal Process and Markov Renewal Function Key Renewal Theorem for MRPs Extended Key Renewal Theorem Semi-Markov Processes: Further Results Markov Regenerative Processes Applications to Queues Diffusion Processes Brownian Motion Sample Path Properties of BM Kolmogorov Equations for Standard Brownian Motion First Passage Times Reflected SBM Reflected BM and Limiting Distributions BM and Martingales Cost/Reward Models Stochastic Integration Stochastic Differential Equations Applications to Finance Epilogue Appendix A: Probability of Events Appendix B: Univariate Random Variables Appendix C: Multivariate Random Variables Appendix D: Generating Functions Appendix E: Laplace-Stieltjes Transforms Appendix F: Laplace Transforms Appendix G: Modes of Convergence Appendix H: Results from Analysis Appendix I: Difference and Differential Equations Answers to Selected Problems References Index Exercises appear at the end of each chapter.