Probability and Random Processes

A resource for probability AND random processes, with hundreds of worked examples and probability and Fourier transform tables This survival guide in probability and random processes eliminates the need to pore through several resources to find a certain formula or table. It offers a compendium of most distribution functions used by communication engineers, queuing theory specialists, signal processing engineers, biomedical engineers, physicists, and students. Key topics covered include: Random variables and most of their frequently used discrete and continuous probability distribution functions Moments, transformations, and convergences of random variables Characteristic, generating, and moment-generating functions Computer generation of random variates Estimation theory and the associated orthogonality principle Linear vector spaces and matrix theory with vector and matrix differentiation concepts Vector random variables Random processes and stationarity concepts Extensive classification of random processes Random processes through linear systems and the associated Wiener and Kalman filters Application of probability in single photon emission tomography (SPECT) More than 400 figures drawn to scale assist readers in understanding and applying theory. Many of these figures accompany the more than 300 examples given to help readers visualize how to solve the problem at hand. In many instances, worked examples are solved with more than one approach to illustrate how different probability methodologies can work for the same problem. Several probability tables with accuracy up to nine decimal places are provided in the appendices for quick reference. A special feature is the graphical presentation of the commonly occurring Fourier transforms, where both time and frequency functions are drawn to scale. This book is of particular value to undergraduate and graduate students in electrical, computer, and civil engineering, as well as students in physics and applied mathematics. Engineers, computer scientists, biostatisticians, and researchers in communications will also benefit from having a single resource to address most issues in probability and random processes.

Table of Contents:
Preface. Chapter 1: Sets, Fields, and Events. 1.1 Set Definitions. 1.2 Set Operations. 1.3 Set Algebras, Fields, and Events. Chapter 2: Probability Space and Axioms. 2.1 Probability Space. 2.2 Conditional Probability. 2.3 Independence. 2.4 Total Probability and Bayes' Theorem. Chapter 3: Basic Combinatorics. 3.1 Basic Counting Principles. 3.2 Permutations. 3.3 Combinations. Chapter 4: Discrete Distributions. 4.1 Bernoulli Trials. 4.2 Binomial Distribution. 4.3 Multinomial Distribution. 4.4 Geometric Distribution. 4.5 Negative Binomial Distribution. 4.6 Hypergeometric Distribution. 4.7 Poisson Distribution. 4.8 Logarithmic Distribution. 4.9 Summary of Discrete Distributions. Chapter 5: Random Variables. 5.1 Definition of Random Variables. 5.2 Determination of Distribution and Density Functions. 5.3 Properties of Distribution and Density Functions. 5.4 Distribution Functions from Density Functions. Chapter 6: Continuous Random Variables and Basic Distributions. 6.1 Introduction. 6.2 Uniform Distribution. 6.3 Exponential Distribution. 6.4 Normal or Gaussian Distribution. Chapter 7: Other Continuous Distributions. 7.1 Introduction. 7.2 Triangular Distribution. 7.3 Laplace Distribution. 7.4 Erlang Distribution. 7.5 Gamma Distribution. 7.6 Weibull Distribution. 7.7 Chi-Square Distribution. 7.8 Chi and Other Allied Distributions. 7.9 Student-t Density. 7.10 Snedecor F Distribution. 7.11 Lognormal Distribution. 7.12 Beta Distribution. 7.13 Cauchy Distribution. 7.14 Pareto Distribution. 7.15 Gibbs Distribution. 7.16 Mixed Distributions. 7.17 Summary of Distributions of Continuous Random Variables. Chapter 8: Conditional Densities and Distributions. 8.1 Conditional Distribution and Density for P(A) = 0. 8.2 Conditional Distribution and Density for P(A) 0. 8.3 Total Probability and Bayes' Theorem for Densities. Chapter 9: Joint Densities and Distributions. 9.1 Joint Discrete Distribution Functions. 9.2 Joint Continuous Distribution Functions 9.3 Bivariate Gaussian Distributions. Chapter 10: Moments and Conditional Moments. 10.1 Expectations. 10.2 Variance. 10.3 Means and Variances of Some Distributions. 10.4 Higher-Order Moments. 10.5 Bivariate Gaussian. Chapter 11: Characteristic Functions and Generating Functions. 11.1 Characteristic Functions. 11.2 Examples of Characteristic Functions. 11.3 Generating Functions. 11.4 Examples of Generating Functions. 11.5 Moment Generating Functions. 11.6 Cumulant Generating Functions. 11.7 Table of Means and Variances. Chapter 12: Functions of a Single Random Variable. 12.1 Random Variable g(X). 12.2 Distribution of Y = g(X ). 12.3 Direct Determination of Density fY (y) from fX(x). 12.4 Inverse Problem: Finding g(x) Given fX(x) and fY (y). 12.5 Moments of a Function of a Random Variable. Chapter 13: Functions of Multiple Random Variables. 13.1 Function of Two Random Variables, Z = g(X,Y ). 13.2 Two Functions of Two Random Variables, Z = g(X,Y ), W = h(X,Y ). 13.3 Direct Determination of Joint Density fZW(z,w ) from fXY(x,y). 13.4 Solving Z = g(X,Y ) Using an Auxiliary Random Variable. 13.5 Multiple Functions of Random Variables. Chapter 14: Inequalities, Convergences, and Limit Theorems. 14.1 Degenerate Random Variables. 14.2 Chebyshev and Allied Inequalities. 14.3 Markov Inequality. 14.4 Chernoff Bound. 14.5 Cauchy-Schwartz Inequality. 14.6 Jensen's Inequality. 14.7 Convergence Concepts. 14.8 Limit Theorems. Chapter 15: Computer Methods for Generating Random Variates. 15.1 Uniform-Distribution Random Variates. 15.2 Histograms. 15.3 Inverse Transformation Techniques. 15.4 Convolution Techniques. 15.5 Acceptance-Rejection Techniques. Chapter 16: Elements of Matrix Algebra. 16.1 Basic Theory of Matrices. 16.2 Eigenvalues and Eigenvectors of Matrices. 16.3 Vectors and Matrix Differentiations. 16.4 Block Matrices. Chapter 17: Random Vectors and Mean-Square Estimation. 17.1 Distributions and Densities. 17.2 Moments of Random Vectors. 17.3 Vector Gaussian Random Variables. 17.4 Diagonalization of Covariance Matrices. 17.5 Simultaneous Diagonalization of Covariance Matrices. 17.6 Linear Estimation of Vector Variables. Chapter 18: Estimation Theory. 18.1 Criteria of Estimators. 18.2 Estimation of Random Variables. 18.3 Estimation of Parameters (Point Estimation). 18.4 Interval Estimation (Confidence Intervals). 18.5 Hypothesis Testing (Binary). 18.6 Bayesian Estimation. Chapter 19: Random Processes. 19.1 Basic Definitions. 19.2 Stationary Random Processes. 19.3 Ergodic Processes. 19.4 Estimation of Parameters of Random Processes. 19.5 Power Spectral Density. Chapter 20: Classification of Random Processes. 20.1 Specifications of Random Processes. 20.2 Poisson Process. 20.3 Binomial Process. 20.4 Independent Increment Process. 20.5 Random-Walk Process. 20.6 Gaussian Process. 20.7 Wiener Process (Brownian Motion). 20.8 Markov Process. 20.9 Markov Chain. 20.10 Martingale Process. 20.11 Periodic Random Process. 20.12 Aperiodic Random Process (Karhunen-Loeve Expansion). Chapter 21: Random Processes and Linear Systems. 21.1 Review of Linear Systems. 21.2 Random Processes through Linear Systems. 21.3 Linear Filters. 21.4 Bandpass Stationary Random Processes. Chapter 22: Weiner and Kalman Filters. 22.1 Review of Orthogonality Principle. 22.2 Wiener Filtering. 22.3 Discrete Kalman Filter. 22.4 Continuous Kalman Filter. Chapter 23: Probabilistic Methods in Transmission Tomography. 23.1 Introduction. 23.2 Stochastic Model. 23.3 Stochastic Estimation Algorithm. 23.4 Prior Distribution P(M). 23.5 Computer Simulation. 23.6 Results and Conclusions. 23.7 Discussion of Results. 23.8 References for Chapter 23. APPENDIXES. A: A Fourier Transform Tables. B: Cumulative Gaussian Tables. C: Inverse Cumulative Gaussian Tables. D: Inverse Chi-Square Tables. E: Inverse Student-t Tables. F: Cumulative Poisson Distribution. G: Cumulative Binomial Distribution. References. Index.