Probabilistic Numerics: Computation as Machine Learning

Probabilistic numerical computation formalises the connection between machine learning and applied mathematics. Numerical algorithms approximate intractable quantities from computable ones. They estimate integrals from evaluations of the integrand, or the path of a dynamical system described by differential equations from evaluations of the vector field. In other words, they infer a latent quantity from data. This book shows that it is thus formally possible to think of computational routines as learning machines, and to use the notion of Bayesian inference to build more flexible, efficient, or customised algorithms for computation. The text caters for Masters' and PhD students, as well as postgraduate researchers in artificial intelligence, computer science, statistics, and applied mathematics. Extensive background material is provided along with a wealth of figures, worked examples, and exercises (with solutions) to develop intuition.

Table of Contents:
Introduction; 1. Mathematical background; 2. Integration; 3. Linear algebra; 4. Local optimisation; 5. Global optimisation; 6. Solving ordinary differential equations; 7. The frontier; Solutions to exercises; References; Index.

About the Author :
Philipp Hennig holds the Chair for the Methods of Machine Learning at the University of Tübingen, and an adjunct position at the Max Planck Institute for Intelligent Systems. He has dedicated most of his career to the development of Probabilistic Numerical Methods. Hennig's research has been supported by Emmy Noether, Max Planck and ERC fellowships. He is a co-Director of the Research Program for the Theory, Algorithms and Computations of Learning Machines at the European Laboratory for Learning and Intelligent Systems (ELLIS). Michael A. Osborne is Professor of Machine Learning at the University of Oxford, and a co-Founder of Mind Foundry Ltd. His research has attracted £10.6M of research funding and has been cited over 15,000 times. He is very, very Bayesian. Hans P. Kersting is a postdoctoral researcher at INRIA and École Normale Supérieure in Paris, working in machine learning with expertise in Bayesian inference, dynamical systems, and optimisation.

Review :
'Computational methods for solving numerical problems lie at the heart of many of the technological advances in science and engineering over the last five decades, and underpin fields as diverse as artificial intelligence, climate modelling, and epidemiology. This impressive text rethinks numerical problems through the lens of probabilistic inference and decision making. This fresh perspective opens up a new chapter in this field, and suggests new and highly efficient methods. A landmark achievement!' Zoubin Ghahramani, University of Cambridge 'This beautiful book is both timely and important with deep roots in powerful early exposition in numerical analysis. In this stunning and comprehensive new book, early developments from Kac and Larkin have been comprehensively built upon, formalised and extended by including modern day machine learning, numerical analysis and the formal Bayesian statistical methodology. Probabilistic Numerical methodology is of enormous importance for this age of data-centric science and Hennig, Osborne and Kersting are to be congratulated in providing us with this definitive volume.' Mark Girolami, University of Cambridge and The Alan Turing Institute 'Numerical analysis is at the very heart of digital computing: every result of a computation on a digital computer is a only finite-precision representation of the true mathematical quantity where the precision is the tradeoff between computation time and accuracy. This book presents an in-depth overview of both the past and present of the newly emerging area of probabilistic numerics, where recent advances in probabilistic machine learning are used to develop principled improvements which are both faster and more accurate than classical numerical analysis algorithms. A must-read for every algorithm developer and practitioner in optimization!' Ralf Herbrich, Hasso Plattner Institute 'Probabilistic Numerics spans from the intellectual fireworks of the dawn of a new field to its practical algorithmic consequences. It is precise but accessible and rich in wide-ranging, principled examples. This convergence of ideas from diverse fields in lucid style is the very fabric of good science.' Carl Edward Rasmussen, University of Cambridge 'An important read for anyone who has thought about uncertainty in numerical methods; an essential read for anyone who hasn't …' John Cunningham, Columbia University 'This is a rare example of a textbook that essentially founds a new field, re-casting numerics on stronger, more general foundations. A tour de force.' David Duvenaud, University of Toronto 'The idea of applying probabilistic inference to the problem of numerical analysis must appear bold, possibly outrageous, even to an entrenched Bayesian statistician. Many in machine learning are now familiar with the application of Bayesian methods to problems that involve randomness, say, the estimation of quantities from noisy data. But to apply the 'calculus of uncertainty' to unknown mathematical facts, where the uncertainty arises only from our lack of knowledge, opens up a universe of new possibilities. This elegant idea is at the core of Probabilistic Numerics, and the authors succeed in demonstrating its potential to transform the way we think about computation itself. And that's not even considering what would happen if we were to apply probabilistic numerics to the numerical problems that arise from probabilistic numerics itself!' Thore Graepel, Senior Vice President, Altos Labs '… the machine learning background of the authors comes through clearly in the book … I thoroughly recommend it.' Chris J. Oates, SIAM Review