Numerical Methods for Least Square Problems

The method of least squares was discovered by Gauss in 1795. It has since become the principal tool for reducing the influence of errors when fitting models to given observations. Today, applications of least squares arise in a great number of scientific areas, such as statistics, geodetics, signal processing, and control. In the last 20 years there has been a great increase in the capacity for automatic data capturing and computing. Least squares problems of large size are now routinely solved. Tremendous progress has been made in numerical methods for least squares problems, in particular for generalized and modified least squares problems and direct and iterative methods for sparse problems. Until now there has not been a monograph that covers the full spectrum of relevant problems and methods in least squares. This volume gives an in-depth treatment of topics such as methods for sparse least squares problems, iterative methods, modified least squares, weighted problems, and constrained and regularized problems. The more than 800 references provide a comprehensive survey of the available literature on the subject. Special Features: Discusses recent methods, many of which are still described only in the research literature. Provides a comprehensive up-to-date survey of problems and numerical methods in least squares computation and their numerical properties. Collects recent research results and covers methods for treating very large and sparse problems with both direct and iterative methods. Covers updating of solutions and factorizations as well as methods for generalized and constrained least squares problems. A solid understanding of numerical linear algebra is needed for the more advanced sections. However, many of the chapters are more elementary and because basic facts and theorems are given in an introductory chapter, the book is partly self-contained.

Table of Contents:
Preface Chapter 1: MATHEMATICAL AND STATISTICAL PROPERTIES OF LEAST SQUARES SOLUTIONS. Introduction The Singular Value Decomposition The QR Decomposition Sensitivity of Least Squares Solutions Chapter 2: BASIC NUMERICAL METHODS. Basics of Floating Point Computation The Method of Normal Equations Elementary Orthogonal Transformations Methods Based on the QR Decomposition Methods Based on Gaussian Elimination Computing the SVD Rank Deficient and Ill-Conditioned Problems Estimating Condition Numbers and Errors Iterative Refinement Chapter 3: MODIFIED LEAST SQUARES PROBLEMS. Introduction Modifying the Full QR Decomposition Downdating the Cholesky Factorization Modifying the Singular Value Decomposition Modifying Rank Revealing QR Decompositions Chapter 4: GENERALIZED LEAST SQUARES PROBLEMS. Generalized QR Decompositions The Generalized SVD General Linear Models and Generalized Least Squares Weighted Least Squares Problems Minimizing the $l_p$ Norm Total Least Squares Chapter 5: CONSTRAINED LEAST SQUARES PROBLEMS. Linear Equality Constraints Linear Inequality Constraints Quadratic Constraints Chapter 6: DIRECT METHODS FOR SPARSE PROBLEMS. Introduction Banded Least Squares Problems Block Angular Least Squares Problems Tools for General Sparse Problems Fill Minimizing Column Orderings The Numerical Cholesky and QR-Decompositions Special Topics Sparse Constrained Problems Software and Test Results Chapter 7: ITERATIVE METHODS FOR LEAST SQUARES PROBLEMS. Introduction Basic Iterative Methods Block Iterative Methods Conjugate Gradient Methods Incomplete Factorization Preconditioners Methods Based on Lanczos Bidiagonalization Methods for Constrained Problems Chapter 8: LEAST SQUARES PROBLEMS WITH SPECIAL BASES. Least Squares Approximation and Orthogonal Systems Polynomial Approximation Discrete Fourier Analysis Toeplitz Least Squares Problems Kronecker Product Problems Chapter 9: NONLINEAR LEAST SQUARES PROBLEMS. The Nonlinear Least Squares Problem Gauss--Newton-Type Methods Newton Type Methods Separable and Constrained Problems Bibliography Index.

Review :
'...This book, which was written by a celebrated person in the field, has become a monumental work ... Altogether, very clearly written and a must for everyone who is interested in least squares, as well as all mathematics libraries.' Bob Matheij, ITW Nieuws '...Until now there has not been a monograph covering the full spectrum of methods and applications in least squares. ...This book is a great masterpiece that will serve as a reference for many years. It must be on the shelves of every numerical analyst, computational scientist and engineer, statistician and electrical engineer.' Claude Brejinski, Numerical Algorithms 'This monograph covers the full spectrum of relevant problems and up-to-date methods in least squares ... a milestone in numerical linear algebra ...' H. Spath, Zentrallblatt fur Mathematik ' ... The author has done a superb job of including the most recent research results, many of which are presented here for the first time in book form. The bibliography is comprehensive and contains more than eight hundred entries. The emphasis of the book is on linear least squares problems, but it also contains a chapter on surveying numerical methods for nonlinear problems.' Hongyuan Zha, Mathematical Reviews 'This book gives a very broad coverage of linear least squares problems. Detailed descriptions are provided for the best algorithms to use and the current literature, with some identification of software availability. No examples are given, and there are few graphs, but the detailed information about methods and algorithms makes this an excellent book ... If you are going to solve a least squares problem of any magnitude, you need Numerical Methods for Least Squares Problems ...' B. A. Finlayson, Applied Mechanics Review "A comprehensive and up-to-date treatment that includes many recent developments. In addition to basic methods, it covers methods for modified and generalized least squares problems, and direct and iterative methods for sparse problems." -Arnold M. Ostebee, The American Mathematical Monthly, January 1997. "Bjorck is an expert on least squares problems...This volume surveys numerical methods for these problems. ...its strength is in the detailed discussion of least squares problems and of their various solution techniques." -B. Borchers, CHOICE, Vol. 34, No. 3, November 1996.