Data Analysis in High Energy Physics: A Practical Guide to Statistical Methods

This practical guide covers the essential tasks in statistical data analysis encountered in high energy physics and provides comprehensive advice for typical questions and problems. The basic methods for inferring results from data are presented as well as tools for advanced tasks such as improving the signal-to-background ratio, correcting detector effects, determining systematics and many others. Concrete applications are discussed in analysis walkthroughs. Each chapter is supplemented by numerous examples and exercises and by a list of literature and relevant links. The book targets a broad readership at all career levels - from students to senior researchers. An accompanying website provides more algorithms as well as up-to-date information and links. * Free solutions manual available for lecturers at www.wiley-vch.de/supplements/

Table of Contents:
Preface XV List of Contributors XIX 1 Fundamental Concepts 1 Roger Barlow 1.1 Introduction 1 1.2 Probability Density Functions 2 1.3 Theoretical Distributions 5 1.4 Probability 16 1.5 Inference and Measurement 20 1.6 Exercises 24 2 Parameter Estimation 27 Olaf Behnke and Lorenzo Moneta 2.1 Parameter Estimation in High Energy Physics: IntroductoryWords 27 2.2 Parameter Estimation: Definition and Properties 27 2.3 The Method of Maximum Likelihood 29 2.4 The Method of Least Squares 40 2.5 Maximum-Likelihood Fits: Unbinned, Binned, Standard and Extended Likelihood 52 2.6 Bayesian Parameter Estimation 67 2.7 Exercises 69 3 Hypothesis Testing 75 Gregory Schott 3.1 Basic Concepts 75 3.2 Choosing the Test Statistic 80 3.3 Choice of the Critical Region 82 3.4 Determining Test Statistic Distributions 82 3.5 p-Values 83 3.6 Inversion of Hypothesis Tests 89 3.7 Bayesian Approach to Hypothesis Testing 92 3.8 Goodness-of-Fit Tests 92 3.9 Conclusion 102 3.10 Exercises 102 4 Interval Estimation 107 Luc Demortier 4.1 Introduction 107 4.2 Characterisation of Interval Constructions 108 4.3 Frequentist Methods 110 4.4 Bayesian Methods 133 4.5 Graphical Comparison of Interval Constructions 140 4.6 The Role of Intervals in Search Procedures 142 4.7 Final Remarks and Recommendations 146 4.8 Exercises 146 5 Classification 153 Helge Voss 5.1 Introduction to Multivariate Classification 153 5.2 Classification from a Statistical Perspective 155 5.3 Multivariate Classification Techniques 162 5.4 General Remarks 182 5.6 Exercises 184 6 Unfolding 187 Volker Blobel 6.1 Inverse Problems 187 6.2 Solution with Orthogonalisation 196 6.3 Regularisation Methods 203 6.4 The Discrete Cosine Transformation and Projection Methods 209 6.5 Iterative Unfolding 213 6.6 Unfolding Problems in Particle Physics 215 6.7 Programs Used for Unfolding in High Energy Physics 221 6.8 Exercise 223 7 ConstrainedFits 227 Benno List 7.1 Introduction 227 7.2 Solution by Elimination 230 7.3 The Method of Lagrange Multipliers 232 7.4 The Lagrange Multiplier Problem with Linear Constraints and Quadratic Objective Function 237 7.5 Iterative Solution of the Lagrange Multiplier Problem 244 7.6 Further Reading and Web Resources 259 7.7 Exercises 260 8 How to Deal with Systematic Uncertainties 263 Rainer Wanke 8.1 Introduction 263 8.2 What Are Systematic Uncertainties? 264 8.3 Detection of Possible Systematic Uncertainties 265 8.4 Estimation of Systematic Uncertainties 272 8.5 How to Avoid Systematic Uncertainties 288 8.6 Conclusion 293 8.7 Exercise 295 9 Theory Uncertainties 297 Markus Diehl 9.1 Overview 297 9.2 Factorisation: A Cornerstone of Calculations in QCD 298 9.3 Power Corrections 308 9.4 The Final State 310 9.5 From Hadrons to Partons 314 9.6 Exercises 324 10 Statistical Methods Commonly Used in High Energy Physics 329 Carsten Hensel and Kevin Kroninger 10.1 Introduction 329 10.2 Estimating Efficiencies 329 10.3 Estimating the Contributions of Processes to a Dataset: The Matrix Method 334 10.4 Estimating Parameters by Comparing Shapes of Distributions: The Template Method 337 10.5 Ensemble Tests 345 10.6 The Experimenter's Role and Data Blinding 351 10.7 Exercises 354 11 Analysis Walk-Throughs 357 Aart Heijboer and Ivo van Vulpen 11.1 Introduction 357 11.2 Search for a Z0 Boson Decaying into Muons 357 11.3 Measurement 369 11.4 Exercises 377 12 Applications in Astronomy 381 Harrison B. Prosper 12.1 Introduction 381 12.2 A Survey of Applications 382 12.3 Nested Sampling 401 12.4 Outlook and Conclusions 404 12.5 Exercises 405 References 405 The Authors 409

About the Author :
Olaf Behnke is a staff physicist at DESY (Hamburg) and an expert for the physics of heavy quarks. He currently holds the position of physics chair in the ZEUS experiment and has recently also joined CMS. Kevin Kroninger is working at the University of Gottingen on the ATLAS experiment, with a focus on top-quark physics. He is one of the authors of the Bayesian Analysis Toolkit (BAT). Thomas Schorner-Sadenius is a staff scientiest at DESY, Hamburg, where he is currently the leader of the Analysis Centre of the German Helmholtz Alliance ?Physics at the Terascale?. Gregory Schott is a physicist employed at the Karlsruhe Institute of Technology and works on the CMS experiment. His main contributions are in the fi eld of Higgs physics. He is one of the authors of the ROOSTATS software package.