Numerical Bayesian Methods Applied to Signal Processing: (Statistics and Computing)

This book is concerned with the processing of signals that have been sampled and digitized. The authors present algorithms for the optimization, random simulation, and numerical integration of probability densities for applications of Bayesian inference to signal processing. In particular, methods are developed for the computation of marginal densities and evidence, and are applied to previously intractable problems either involving large numbers of parameters or where the signal model is of a complex form. The emphasis is on the applications of these methods notably to the restoration of digital audio recordings and biomedical data. After a chapter which sets out the main principles of Bayesian inference applied to signal processing, subsequent chapters cover numerical approaches to these techniques, the use of Markov chain Monte Carlo methods, the identification of abrupt changes in data using the Bayesian piecewise linear model, and identifying missing samples in digital audio signals.

Table of Contents:
1 Introduction.- 2 Probabilistic Inference in Signal Processing.- 2.1 Introduction.- 2.2 The likelihood function.- 2.3 Bayesian data analysis.- 2.4 Prior probabilities.- 2.5 The removal of nuisance parameters.- 2.6 Model selection using Bayesian evidence.- 2.7 The general linear model.- 2.8 Interpretations of the general linear model.- 2.9 Example of marginalization.- 2.10 Example of model selection.- 2.11 Concluding remarks.- 3 Numerical Bayesian Inference.- 3.1 The normal approximation.- 3.2 Optimization.- 3.3 Integration.- 3.4 Numerical quadrature.- 3.5 Asymptotic approximations.- 3.6 The Monte Carlo method.- 3.7 The generation of random variates.- 3.8 Evidence using importance sampling.- 3.9 Marginal densities.- 3.10 Opportunities for variance reduction.- 3.11 Summary.- 4 Markov Chain Monte Carlo Methods.- 4.1 Introduction.- 4.2 Background on Markov chains.- 4.3 The canonical distribution.- 4.4 The Gibbs sampler.- 4.5 The Metropolis-Hastings algorithm.- 4.6 Dynamical sampling methods.- 4.7 Implementation of simulated annealing.- 4.8 Other issues.- 4.9 Free energy estimation.- 4.10 Summary.- 5 Retrospective Changepoint Detection.- 5.1 Introduction.- 5.2 The simple Bayesian step detector.- 5.3 The detection of changepoints using the general linear model.- 5.4 Recursive Bayesian estimation.- 5.5 Detection of multiple changepoints.- 5.6 Implementation details.- 5.7 Multiple changepoint results.- 5.8 Concluding Remarks.- 6 Restoration of Missing Samples in Digital Audio Signals.- 6.1 Introduction.- 6.2 Model formulation.- 6.3 The EM algorithm.- 6.4 Gibbs sampling.- 6.5 Implementation issues.- 6.6 Relationship between the three restoration methods.- 6.7 Simulations.- 6.8 Discussion.- 6.9 Concluding remarks.- 7 Integration in Bayesian Data Analysis.- 7.1 Polynomial data.-7.2 Decay problem.- 7.3 General model selection.- 7.4 Summary.- 8 Conclusion.- 8.1 A review of the work.- 8.2 Further work.- A The General Linear Model.- A.1 Integrating out model amplitudes.- A.1.1 Least squares.- A.1.2 Orthogonalization.- A.2 Integrating out the standard deviation.- A.3 Marginal density for a linear coefficient.- A.4 Marginal density for standard deviation.- A.5 Conditional density for a linear coefficient.- A.6 Conditional density for standard deviation.- B Sampling from a Multivariate Gaussian Density.- C Hybrid Monte Carlo Derivations.- C.1 Full Gaussian likelihood.- C.2 Student-t distribution.- C.3 Remark.- D EM Algorithm Derivations.- D.l Expectation.- D.2 Maximization.- E Issues in Sampling Based Approaches to Integration.- E.1 Marginalizing using the conditional density.- E.2 Approximating the conditional density.- E.3 Gibbs sampling from the joint density.- E.4 Reverse importance sampling.- F Detailed Balance.- F.1 Detailed balance in the Gibbs sampler.- F.2 Detailed balance in the Metropolis Hastings algorithm..- F.3 Detailed balance in the Hybrid Monte Carlo algorithm..- F.4 Remarks.- References.