A Distribution-Free Theory of Nonparametric Regression: (Springer Series in Statistics)

The regression estimation problem has a long history. Already in 1632 Galileo Galilei used a procedure which can be interpreted as ?tting a linear relationship to contaminated observed data. Such ?tting of a line through a cloud of points is the classical linear regression problem. A solution of this problem is provided by the famous principle of least squares, which was discovered independently by A. M. Legendre and C. F. Gauss and published in 1805 and 1809, respectively. The principle of least squares can also be applied to construct nonparametric regression estimates, where one does not restrict the class of possible relationships, and will be one of the approaches studied in this book. Linear regression analysis, based on the concept of a regression function, was introduced by F. Galton in 1889, while a probabilistic approach in the context of multivariate normal distributions was already given by A. B- vais in 1846. The ?rst nonparametric regression estimate of local averaging type was proposed by J. W. Tukey in 1947. The partitioning regression - timate he introduced, by analogy to the classical partitioning (histogram) density estimate, can be regarded as a special least squares estimate.

Table of Contents:
Why Is Nonparametric Regression Important?.- How to Construct Nonparametric Regression Estimates?.- Lower Bounds.- Partitioning Estimates.- Kernel Estimates.- k-NN Estimates.- Splitting the Sample.- Cross-Validation.- Uniform Laws of Large Numbers.- Least Squares Estimates I: Consistency.- Least Squares Estimates II: Rate of Convergence.- Least Squares Estimates III: Complexity Regularization.- Consistency of Data-Dependent Partitioning Estimates.- Univariate Least Squares Spline Estimates.- Multivariate Least Squares Spline Estimates.- Neural Networks Estimates.- Radial Basis Function Networks.- Orthogonal Series Estimates.- Advanced Techniques from Empirical Process Theory.- Penalized Least Squares Estimates I: Consistency.- Penalized Least Squares Estimates II: Rate of Convergence.- Dimension Reduction Techniques.- Strong Consistency of Local Averaging Estimates.- Semirecursive Estimates.- Recursive Estimates.- Censored Observations.- Dependent Observations.

Review :
From the reviews: MATHEMATICAL REVIEWS "…this book is written by a highly competent, international team of researchers, who have made important contributions and have had a large impact on the field of nonparametric regression estimation and are still active in different subfields; the monograph is almost self-contained and written in such a way that it is a valuable resource for both the researches and graduate students who are novices in the field. The above is possible thanks to the authors’ clear and systematic way of presenting fundamental results and their proofs…this book is a valuable source of mathematical techniques and provides systematic in-depth analysis of nonparametric regression with random design. Although this research monograph reflects recent studies in the field, it can also serve as an encyclopedia of nonparametric regression estimation." SHORT BOOK REVIEWS "The book gives a deep and modern mathematical treatment of nonparametric regression with random design. From the table of contents it is seen that all well-known classes of estimators are dealt with. For each of them, the authors mainly prove results on consistency and on rates of convergence. The book follow the style Theorem-Proof and gives rigorous derivations of all the results. There is a useful mathematical appendix with proofs and exponential type inequalities for sums of independent variables and for sum of martingale differences. Each chapter has a section called "Bibliographic Notes" containing references to the extensive bibliography of more than 400 items. Each chapter ends with a  number of problems and exercises, which could be used in a teaching situation." "This book is on nonparametric regression with random designs. … The text … is self contained and suitable for students and researchers in statistics, mathematics, computer science and engineering. … This is a definitive treatise on the important methods ofestimation in nonparametric regression and provides a clear exposition of the issues involved in consistency, rate of convergence and asymptotic optimality of different classes of estimates. A must have book." (Arup Bose, Sankhya, Vol. 65 (2), 2003) "If you choose to browse this book from page to page you will find a stunning amount of details … . important topics such as penalized squares, spline estimates or recursive estimators are … discussed in much detail. … I think this is an excellent book … both for research students and teachers. … The educational value of the book is enhanced by the presentation of some classical mathematical masterpieces. … I strongly recommend to get this book … ." (László Gerencsér, Journal of Applied Statistics, 2005) "The authors of this book endeavored to present a ‘modern approach’ to non-parametric regression … . This book is excellent as a reference, because the proofs are written in an extremely clear manner and the topics selected are discussed very clearly and are interesting. … I would recommend it to graduate students who want to know how certain results are obtained. … an interesting collection of topics and results not replicated by others." (Kathryn A. Prewitt, Journal of the American Statistical Association, 2004) "This book presents a modern approach to nonparametric regression estimation with random design. … This book is a self-contained text, intended for a wide audience, including graduate students in statistics, mathematics and computer sciences and researchers. Because of the clear mathematical presentation it can be used also for a course on nonparametric regression estimation. … This makes the book a valuable reference for anyone interested in nonparametric regression as well as a source of many useful mathematical techniques." (H. Liero, Zentralblatt MATH, Vol. 1021, 2003) "This book presents an impressive overview of many nonparametric regressionmethods. … Altogether, the book is almost ideally self-contained … . It is clearly written, and presents a wealth of popular statistical methods relevant in many application areas. Hence, it will be an often consulted book in the academic library, and a good source on which to build a lecture course. … Academic mathematical statisticians will … also find the book of great use … ." (Dr. F. P. A. Coolen, Kwantitatieve Methoden, Issue 70B38, 2003) "The monograph under review can be considered as the next volume in a series of seminal monographs on the theoretical foundations of nonparametric estimation … . the monograph is almost self-contained and written in such a way that it is a valuable resource for both the researchers and graduate students who are novices in the field. The above is possible thanks to the authors’ clear and systematic way of presenting fundamental results and their proofs. … a valuable source of mathematical techniques … ." (Ewaryst Rafajlowicz, Mathematical Reviews, 2003 g) "The book gives a deep and modern mathematical treatment of nonparametric regression with random design. … all well-known classes of estimators are dealt with. For each of them, the authors mainly prove results on consistency and on rates of convergence. The book follows the style Theorem-Proof and gives rigorous derivations of all the results. There is a useful mathematical appendix with proofs of exponential type inequalities for sums of independent variables and for sums of martingale differences." (N. D. C. Veraverbeke, Short Book Reviews, Vol. 23 (2), 2003)