Mathematical Statistics and Stochastic Processes

Generally, books on mathematical statistics are restricted to the case of independent identically distributed random variables. In this book however, both this case AND the case of dependent variables, i.e. statistics for discrete and continuous time processes, are studied. This second case is very important for today’s practitioners. Mathematical Statistics and Stochastic Processes is based on decision theory and asymptotic statistics and contains up-to-date information on the relevant topics of theory of probability, estimation, confidence intervals, non-parametric statistics and robustness, second-order processes in discrete and continuous time and diffusion processes, statistics for discrete and continuous time processes, statistical prediction, and complements in probability. This book is aimed at students studying courses on probability with an emphasis on measure theory and for all practitioners who apply and use statistics and probability on a daily basis.

Table of Contents:
Preface xiii PART 1. MATHEMATICAL STATISTICS 1 Chapter 1. Introduction to Mathematical Statistics 3 1.1. Generalities 3 1.2. Examples of statistics problems 4 Chapter 2. Principles of Decision Theory 9 2.1. Generalities 9 2.2. The problem of choosing a decision function 11 2.3. Principles of Bayesian statistics 13 2.4. Complete classes 17 2.5. Criticism of decision theory – the asymptotic point of view 18 2.6. Exercises 18 Chapter 3. Conditional Expectation 21 3.1. Definition 21 3.2. Properties and extension 22 3.3. Conditional probabilities and conditional distributions 24 3.4. Exercises 26 Chapter 4. Statistics and Sufficiency 29 4.1. Samples and empirical distributions 29 4.2. Sufficiency 31 4.3. Examples of sufficient statistics – an exponential model 33 4.4. Use of a sufficient statistic 35 4.5. Exercises 36 Chapter 5. Point Estimation 39 5.1. Generalities 39 5.2. Sufficiency and completeness 42 5.3. The maximum-likelihood method 45 5.4. Optimal unbiased estimators 49 5.5. Efficiency of an estimator 56 5.6. The linear regression model 65 5.7. Exercises 68 Chapter 6. Hypothesis Testing and Confidence Regions 73 6.1. Generalities 73 6.2. The Neyman–Pearson (NP) lemma 75 6.3. Multiple hypothesis tests (general methods) 80 6.4. Case where the ratio of the likelihoods is monotonic 84 6.5. Tests relating to the normal distribution 86 6.6. Application to estimation: confidence regions 86 6.7. Exercises 90 Chapter 7. Asymptotic Statistics 101 7.1. Generalities 101 7.2. Consistency of the maximum likelihood estimator 103 7.3. The limiting distribution of the maximum likelihood estimator 104 7.4. The likelihood ratio test 106 7.5. Exercises 108 Chapter 8. Non-Parametric Methods and Robustness 113 8.1. Generalities 113 8.2. Non-parametric estimation 114 8.3. Non-parametric tests 117 8.4. Robustness 121 8.5. Exercises 124 PART 2. STATISTICS FOR STOCHASTIC PROCESSES 131 Chapter 9. Introduction to Statistics for Stochastic Processes 133 9.1. Modeling a family of observations 133 9.2. Processes 134 9.3. Statistics for stochastic processes 137 9.4. Exercises 138 Chapter 10. Weakly Stationary Discrete-Time Processes 141 10.1. Autocovariance and spectral density 141 10.2. Linear prediction and Wold decomposition 144 10.3. Linear processes and the ARMA model 146 10.4. Estimating the mean of a weakly stationary process 149 10.5. Estimating the autocovariance 151 10.6. Estimating the spectral density 151 10.7. Exercises 155 Chapter 11. Poisson Processes – A Probabilistic and Statistical Study 163 11.1. Introduction 163 11.2. The axioms of Poisson processes 164 11.3. Interarrival time 166 11.4. Properties of the Poisson process 168 11.5. Notions on generalized Poisson processes 170 11.6. Statistics of Poisson processes 172 11.7. Exercises 177 Chapter 12. Square-Integrable Continuous-Time Processes 183 12.1. Definitions 183 12.2. Mean-square continuity 183 12.3. Mean-square integration 184 12.4. Mean-square differentiation 187 12.5. The Karhunen–Loeve theorem 188 12.6. Wiener processes 189 12.7. Notions on weakly stationary continuous-time processes 195 12.8. Exercises 197 Chapter 13. Stochastic Integration and Diffusion Processes 203 13.1. Itô integral 203 13.2. Diffusion processes 206 13.3. Processes defined by stochastic differential equations and stochastic integrals 212 13.4. Notions on statistics for diffusion processes 215 13.5. Exercises 216 Chapter 14. ARMA Processes 219 14.1. Autoregressive processes 219 14.2. Moving average processes 223 14.3. General ARMA processes 224 14.4. Non-stationary models 226 14.5. Statistics of ARMA processes 228 14.6. Multidimensional processes 232 14.7. Exercises 233 Chapter 15. Prediction 239 15.1. Generalities 239 15.2. Empirical methods of prediction 240 15.3. Prediction in the ARIMA model 242 15.4. Prediction in continuous time 244 15.5. Exercises 245 PART 3. SUPPLEMENT 249 Chapter 16. Elements of Probability Theory 251 16.1. Measure spaces: probability spaces 251 16.2. Measurable functions: real random variables 253 16.3. Integrating real random variables 255 16.4. Random vectors 259 16.5. Independence 261 16.6. Gaussian vectors 262 16.7. Stochastic convergence 264 16.8. Limit theorems 265 Appendix. Statistical Tables 267 A1.1. Random numbers 267 A1.2. Distribution function of the standard normal distribution 268 A1.3. Density of the standard normal distribution 269 A1.4. Percentiles (tp) of Student’s distribution 270 A1.5. Ninety-fifth percentiles of Fisher–Snedecor distributions 271 A1.6. Ninety-ninth percentiles of Fisher–Snedecor distributions 272 A1.7. Percentiles (χ2 p) of the χ2 distribution with n degrees of freedom 273 A1.8. Individual probabilities of the Poisson distribution 274 A1.9. Cumulative probabilities of the Poisson distribution 275 A1.10. Binomial coefficients Ck n for n ≤ 30 and 0 ≤ k ≤ 7 276 A1.11. Binomial coefficients Ck n for n ≤ 30 and 8 ≤ k ≤ 15 277 Bibliography 279 Index 281