Discrete-time Stochastic Systems: Estimation and Control(Advanced Textbooks in Control and Signal Processing)

Discrete-time Stochastic Systems gives a comprehensive introduction to the estimation and control of dynamic stochastic systems and provides complete derivations of key results such as the basic relations for Wiener filtering. The book covers both state-space methods and those based on the polynomial approach. Similarities and differences between these approaches are highlighted. Some non-linear aspects of stochastic systems (such as the bispectrum and extended Kalman filter) are also introduced and analysed. The books chief features are as follows: inclusion of the polynomial approach provides alternative and simpler computational methods than simple reliance on state-space methods; algorithms for analysis and design of stochastic systems allow for ease of implementation and experimentation by the reader; the highlighting of spectral factorization gives appropriate emphasis to this key concept often overlooked in the literature; explicit solutions of Wiener problems are handy schemes, well suited for computations compared with more commonly available but abstract formulations; complex-valued models that are directly applicable to many problems in signal processing and communications. Changes in the second edition include: additional information covering spectral factorisation and the innovations form; the chapter on optimal estimation being completely rewritten to focus on a posterior estimates rather than maximum likelihood; new material on fixed lag smoothing and algorithms for solving Riccati equations are improved and more up to date; new presentation of polynomial control and new derivation of linear-quadratic-Gaussian control. Discrete-time Stochastic Systems is primarily of benefit to students taking M.Sc. courses in stochastic estimation and control, electronic engineering and signal processing but may also be of assistance for self study and as a reference.

Table of Contents:
1. Introduction.- 1.1 What is a Stochastic System?.- Bibhography.- 2. Some Probability Theory.- 2.1 Introduction.- 2.2 Random Variables and Distributions.- 2.3 Conditional Distributions.- 2.4 The Conditional Mean for Gaussian Variables.- 2.5 Complex-Valued Gaussian Variables.- Exercises.- 3. Models.- 3.1 Introduction.- 3.2 Stochastic Processes.- 3.3 Markov Processes and the Concept of State.- 3.4 Covariance Function and Spectrum.- 3.5 Bispectrum.- 3.A Appendix. Linear Complex-Valued Signals and Systems.- 3.B Appendix. Markov Chains.- Exercises.- 4. Analysis.- 4.1 Introduction.- 4.2 Linear Filtering.- 4.3 Spectral Factorization.- 4.4 Continuous-time Models.- 4.5 Sampling Stochastic Models.- 4.6 The Positive Real Part of the Spectrum.- 4.7 Effect of Linear Filtering on the Bispectrum.- 4.8 Algorithms for Covariance Calculations and Sampling.- 4. A Appendix. Auxiliary Lemmas.- Exercises.- 5. Optimal Estimation.- 5.1 Introduction.- 5.2 The Conditional Mean.- 5.3 The Linear Least Mean Square Estimate.- 5.4 Propagation of the Conditional Probability Density Function.- 5.5 Relation to Maximum Likelihood Estimation.- 5.A Appendix. A Lemma for Optimality of the Conditional Mean.- Exercises.- 6. Optimal State Estimation for Linear Systems.- 6.1 Introduction.- 6.2 The Linear Least Mean Square One-Step Prediction and Filter Estimates.- 6.3 The Conditional Mean.- 6.4 Optimal Filtering and Prediction.- 6.5 Smoothing.- 6.6 Maximum a posteriori Estimates.- 6.7 The Stationary Case.- 6.8 Algorithms for Solving the Algebraic Riccati Equation.- 6.A Appendix. Proofs.- Exercises.- 7. Optimal Estimation for Linear Systems by Polynomial Methods.- 7.1 Introduction.- 7.2 Optimal Prediction.- 7.3 Wiener Filters.- 7.4 Minimum Variance Filters.- 7.5 Robustness Against Modelling Errors.- Exercises.-8. Illustration of Optimal Linear Estimation.- 8.1 Introduction.- 8.2 Spectral Factorization.- 8.3 Optimal Prediction.- 8.4 Optimal Filtering.- 8.5 Optimal Smoothing.- 8.6 Estimation Error Variance.- 8.7 Weighting Pattern.- 8.8 Frequency Characteristics.- Exercises.- 9. Nonlinear Filtering.- 9.1 Introduction.- 9.2 Extended Kaiman Filters.- 9.3 Gaussian Sum Estimators.- 9.4 The Multiple Model Approach.- 9.5 Monte Carlo Methods for Propagating the Conditional Probability Density Functions.- 9.6 Quantized Measurements.- 9.7 Median Filters.- 9.A Appendix. Auxiliary results.- Exercises.- 10. Introduction to Optimal Stochastic Control.- 10.1 Introduction.- 10.2 Some Simple Examples.- 10.3 Mathematical Preliminaries.- 10.4 Dynamic Programming.- 10.5 Some Stochastic Controllers.- Exercises.- 11. Linear Quadratic Gaussian Control.- 11.1 Introduction.- 11.2 The Optimal Controllers.- 11.3 Duality Between Estimation and Control.- 11.4 Closed Loop System Properties.- 11.5 Linear Quadratic Gaussian Design by Polynomial Methods.- 11.6 Controller Design by Linear Quadratic Gaussian Theory.- 11. A Appendix. Derivation of the Optimal Linear Quadratic Gaussian Feedback and the Riccati Equation from the Bellman Equation.- Exercises.- Answers to Selected Exercises.