Assessing and Improving Prediction and Classification

This book begins by presenting methods for performing practical, real-life assessment of the performance of prediction and classification models. It then goes on to discuss techniques for improving the performance of such models by intelligent resampling of training/testing data, combining multiple models into sophisticated committees, and making use of exogenous information to dynamically choose modeling methodologies. Rigorous statistical techniques for computing confidence in predictions and decisions receive extensive treatment. Finally, a hundred pages are devoted to the use of information theory in evaluating and selecting useful predictors. Special attention is paid to Schreiber's Information Transfer, a recent generalization of Grainger Causality. Well commented C++ code is given for every algorithm and technique. The ultimate purpose of this text is three-fold. The first goal is to open the eyes of serious developers to some of the hidden pitfalls that lurk in the model development process. The second is to provide broad exposure for some of the most powerful model enhancement algorithms that have emerged from academia in the last two decades, while not bogging down readers in cryptic mathematical theory. Finally, this text should provide the reader with a toolbox of ready-to-use C++ code that can be easily incorporated into his or her existing programs.

About the Author :
Timothy Masters received a PhD in mathematical statistics with a specialization in numerical computing. Since then he has continuously worked as an independent consultant for government and industry. His early research involved automated feature detection in high-altitude photographs while he developed applications for flood and drought prediction, detection of hidden missile silos, and identification of threatening military vehicles. Later he worked with medical researchers in the development of computer algorithms for distinguishing between benign and malignant cells in needle biopsies. For the last twenty years he has focused primarily on methods for evaluating automated financial market trading systems. He has authored four books on practical applications of neural networks: Practical Neural Network Recipes in C++ (Academic Press, 1993) Signal and Image Processing with Neural Networks (Wiley, 1994) Advanced Algorithms for Neural Networks (Wiley, 1995) Neural, Novel, and Hybrid Algorithms for Time Series Prediction (Wiley, 1995)