Modeling and Analysis of Stochastic Systems, Second Edition: (Chapman & Hall/CRC Texts in Statistical Science)

This practical and accessible text enables students in engineering, business, operations research, public policy and computer science to analyze stochastic systems. Emphasizing the modeling of real-life situations with stochastic elements and analyzing the resulting stochastic model, it presents the major cases of useful stochastic processes-discrete and continuous time Markov chains, renewal processes, regenerative processes, and Markov regenerative processes. The author provides user-friendly, yet rigorous coverage. He demonstrates both numerical and analytical solution methods in detail and includes numerous worked examples and exercises.

Table of Contents:
Introduction What in the World Is a Stochastic Process? How to Characterize a Stochastic Process What Do We Do with a Stochastic Process? Discrete-Time Markov Chains: Transient Behavior Definition and Characterization Examples DTMCs in Other Fields Marginal Distributions Occupancy Times Computation of Matrix Powers DTMCs: First Passage Times Definitions Cumulative Distribution Function of T Absorption Probabilities Expectation of T Generating Function and Higher Moments of T DTMCs: Limiting Behavior Exploring the Limiting Behavior by Examples Irreducibility and Periodicity Recurrence and Transience Determining Recurrence and Transience: Infinite DTMCs Limiting Behavior of Irreducible DTMCs Examples: Limiting Behavior of Infinite State-Space Irreducible DTMCs Limiting Behavior of Reducible DTMCs DTMCs with Costs and Rewards Reversibility Poisson Processes Exponential Distributions Poisson Process: Definitions Event Times in a Poisson Process Superposition and Splitting of Poisson Processes Non-Homogenous Poisson Process Compound Poisson Process Continuous-Time Markov Chains Definitions and Sample Path Properties Examples Transient Behavior: Marginal Distribution Transient Behavior: Occupancy Times Computation of P(t): Finite State-Space Computation of P(t): Infinite State-Space First-Passage Times Exploring the Limiting Behavior by Examples Classification of States Limiting Behavior of Irreducible CTMCs Limiting Behavior of Reducible CTMCs CTMCs with Costs and Rewards Phase-Type Distributions Reversibility Queueing Models Introduction Properties of General Queueing Systems Birth and Death Queues Open Queueing Networks Closed Queueing Networks Single Server Queues Retrial Queue Infinite Server Queue Renewal Processes Introduction Properties of N(t) The Renewal Function Renewal-Type Equation Key Renewal Theorem Recurrence Times Delayed Renewal Processes Alternating Renewal Processes Semi-Markov Processes Renewal Processes with Costs/Rewards Regenerative Processes Markov Regenerative Processes Definitions and Examples Markov Renewal Process and Markov Renewal Function Key Renewal Theorem for MRPs Extended Key Renewal Theorem Semi-Markov Processes: Further Results Markov Regenerative Processes Applications to Queues Diffusion Processes Brownian Motion Sample Path Properties of BM Kolmogorov Equations for Standard Brownian Motion First Passage Times Reflected SBM Reflected BM and Limiting Distributions BM and Martingales Cost/Reward Models Stochastic Integration Stochastic Differential Equations Applications to Finance Epilogue Appendix A: Probability of Events Appendix B: Univariate Random Variables Appendix C: Multivariate Random Variables Appendix D: Generating Functions Appendix E: Laplace-Stieltjes Transforms Appendix F: Laplace Transforms Appendix G: Modes of Convergence Appendix H: Results from Analysis Appendix I: Difference and Differential Equations Answers to Selected Problems References Index Exercises appear at the end of each chapter.

About the Author :
Vidyadhar G. Kulkarni is a Norman Johnson Professor in the Department of Statistics and Operations Research at the University of North Carolina at Chapel Hill.

Review :
This book provides insight on the mathematical methods for modelling and understanding real-world processes. ! for the teacher, this book provides an abundant resource of examples for class exercises and student homework. The modelling examples include many that would be of interest for advanced undergraduate courses on inventory management, reliability, operations management and queuing networks. ! This is a book for the few who would like to know almost everything about stochastic processes and for the many who need to know something, and would like to have their eyes opened to the many models and analytical results available on stochastic processes. --J. Smart, Journal of the Operational Research Society, Vol. 62, 2011 This book is written by a well-experienced scientist and teacher. The material is carefully chosen and presented in a systematic way, starting with the basics and reaching more advanced topics. ! It is very useful to see a large number of examples worked out in detail. Thus, besides university courses, the book can be successfully used for self-education. ! This book is a good example of how to write on sophisticated topics about stochastic processes without involving the heavy measure theoretic machinery. Another advantage of the book is the application-oriented approach that is followed by the author. ! This book can be strongly recommended for introductory, intermediate or advanced university courses on stochastic processes. Both students and their teachers may benefit considerably from it. --Journal of the Royal Statistical Society: Series A, July 2011 ! an accessible, well paced, and very nicely presented book. The publishers are also to be commended on its nice production: it is the sort of book which is a pleasure to read. In all, it is an excellent textbook for use in introductory courses on stochastic processes. --International Statistical Review (2010), 78, 3 This book may be recommended for all readers who want to understand, build and analyse stochastic models and relevant problems. --Zentralblatt MATH 1191 Praise for the First Edition: ! a beautiful introductory textbook on stochastic processes ! Plenty of applications, examples, and exercises bring the reader to the intuitive understanding of the subject. --Zentralblatt MATH Well-chosen examples and interesting exercises make this text a good choice for a first course in stochastic processes for a broad class of students. --Journal of the American Statistical Association