About the Book
Explanatory Model Analysis Explore, Explain and Examine Predictive Models is a set of methods and tools designed to build better predictive models and to monitor their behaviour in a changing environment. Today, the true bottleneck in predictive modelling is neither the lack of data, nor the lack of computational power, nor inadequate algorithms, nor the lack of flexible models. It is the lack of tools for model exploration (extraction of relationships learned by the model), model explanation (understanding the key factors influencing model decisions) and model examination (identification of model weaknesses and evaluation of model's performance). This book presents a collection of model agnostic methods that may be used for any black-box model together with real-world applications to classification and regression problems.
Table of Contents:
I. Introduction 1. Introduction. 2. Model Development. 3. Do-it-yourself. 4. Datasets and models. II. Instance Level. 5. Introduction to Instance-level Exploration. 6. Break-down Plots for Additive Attributions. 7. Break-down Plots for Interactions. 8. Shapley Additive Explanations (SHAP) for Average Attributions. 9. Local Interpretable Model-agnostic Explanations (LIME). 10. Ceteris-paribus Profiles. 11. Ceteris-paribus Oscillations. 12. Local-diagnostics Plots. 13. Summary of Instance-level Exploration. III. Dataset Level. 14. Introduction to Dataset-level Exploration. 15. Model-performance Measures. 16. Variable-importance Measures. 17. Partial-dependence Profiles. 18. Local-dependence and Accumulated-dependence Profiles. 19. Residual Diagnostics Plots. 20. Summary of Model-level Exploration. IV. Use-cases. 21. FIFA 19.
About the Author :
Przemyslaw Biecek is a professor in human-oriented machine learning at the Warsaw University of Technology and Principal Data Scientist in Samsung R&D Institute Poland. His main research project is DrWhy.AI - tools and methods for exploration, explanation, visualisation, and debugging of predictive models.
Tomasz Burzykowski is professor of biostatistics at Hasselt University and Vice-President for Research at International Drug Development Institute (IDDI). He has published extensively on applications of statistics in medicine and biology.
Review :
"The structure is well-conceived, with chapters consisting in five sections: intuition, method, example, pros and cons, and code snippets. I sense a teacher’s long experience behind these choices.
The chapters contain good mathematical detail on the techniques discussed, but the theory is well balanced with examples and code.
The visualizations are great. Often, the gist of a particular technique, and it’s practical, interpretive value, can be gleaned from the visualizations threading through the chapter, along with captions. The authors did a really nice job with this.
The rationale for the book is well-described.
The discussion of techniques seems both comprehensive (given my sense of the field) and helpfully specific, both at the instance and the dataset levels."
-Jeff Webb, University of Utah
"The authors are doing a very good job in addressing the potential readers, by providing a clean presentation and practical guidance on diagnostic graphical tools…Having an ‘intuition section’ at the beginning of each chapter is very useful."
-Riccardo De Bin, University of Oslo
"The book provides a unified presentation of model exploration, visualization, comparison and diagnostics of different machine learning algorithms…This book would be found useful by both students as well as practitioners who analyze their own data. Books including real data examples in R and in Python are needed in this area. (It) will serve as a reference, especially for analyses done with dalex or archivist R package (and )can serve as a textbook of data science courses in many fields including computer science, social sciences, economics and other."
-Patricia Martinkova, Institute of Computer Science of the Czech Academy of Sciences
"There are books that focus on prediction models, for example the element of statistical learning and an introduction to statistical learning but these are not focused on the evaluation of predictive models which is the main focus on the proposed book and its main advantage. As predictive models become very popular in the last years, such a book that focus on the evaluation of the models and model diagnostics can be very popular."
-Ziv Shkedy, Data Science Institute, Hasselt University, Belgium
'The book is clearly and consistently structured and well–written. The graphics are explained conceptually and mathematically. There are chapter sections on the pros and cons of what is proposed, where the authors are generally properly cautious and recommend a mixture of approaches.'
- Antony Unwin, International Statistical Review, 2021 Volume 89, Issue 3
"The structure is well-conceived, with chapters consisting in five sections: intuition, method, example, pros and cons, and code snippets. I sense a teacher’s long experience behind these choices. The chapters contain good mathematical detail on the techniques discussed, but the theory is well balanced with examples and code. The visualizations are great. Often, the gist of a particular technique, and it’s practical, interpretive value, can be gleaned from the visualizations threading through the chapter, along with captions. The authors did a really nice job with this. The rationale for the book is well-described. The discussion of techniques seems both comprehensive (given my sense of the field) and helpfully specific, both at the instance and the dataset levels."
-Jeff Webb, University of Utah
"The authors are doing a very good job in addressing the potential readers, by providing a clean presentation and practical guidance on diagnostic graphical tools…Having an ‘intuition section’ at the beginning of each chapter is very useful."
-Riccardo De Bin, University of Oslo
"The book provides a unified presentation of model exploration, visualization, comparison and diagnostics of different machine learning algorithms…This book would be found useful by both students as well as practitioners who analyze their own data. Books including real data examples in R and in Python are needed in this area. (It) will serve as a reference, especially for analyses done with dalex or archivist R package (and )can serve as a textbook of data science courses in many fields including computer science, social sciences, economics and other."
-Patricia Martinkova, Institute of Computer Science of the Czech Academy of Sciences
"There are books that focus on prediction models, for example the element of statistical learning and an introduction to statistical learning but these are not focused on the evaluation of predictive models which is the main focus on the proposed book and its main advantage. As predictive models become very popular in the last years, such a book that focus on the evaluation of the models and model diagnostics can be very popular."
-Ziv Shkedy, Data Science Institute, Hasselt University, Belgium
"We need to explore the models and learn about their behaviour. This book presents, explains, and summarises the techniques for doing so. Moreover, it provides code in R and Python for doing so. The methods have many similarities with those of sensitivity analysis developed within the Sensitivity Analysis of Model Output (SAMO) community. ... [M]any doctoral students, professional statisticians and researchers should ensure that they have access to it and know how to use its methods when dealing with highly complex functions in their data and model analysis."
-Simon French, in the Journal of the Royal Statistics Society, Series A, June 2022
"The book presents a valuable collection of methods for models’ exploration and diagnostics for various machine learning algorithms. It can be useful in the data and computer science courses for students and instructors, as well as for researchers and practitioners who need to analyze and interpret their statistical and machine learning models both of glass-box and blackbox kind. The book also serves as a great primary for applications of the R and Python software and their packages/libraries, so it is valuable in solving various problems of statistical prediction in various fields."
-Stan Lipovetsky, in Technometrics, July 2022
