An Introduction to Bootstrap Methods with Applications to R

A comprehensive introduction to bootstrap methods in the R programming environment Bootstrap methods provide a powerful approach to statistical data analysis, as they have more general applications than standard parametric methods. An Introduction to Bootstrap Methods with Applications to R explores the practicality of this approach and successfully utilizes R to illustrate applications for the bootstrap and other resampling methods. This book provides a modern introduction to bootstrap methods for readers who do not have an extensive background in advanced mathematics. Emphasis throughout is on the use of bootstrap methods as an exploratory tool, including its value in variable selection and other modeling environments. The authors begin with a description of bootstrap methods and its relationship to other resampling methods, along with an overview of the wide variety of applications of the approach. Subsequent chapters offer coverage of improved confidence set estimation, estimation of error rates in discriminant analysis, and applications to a wide variety of hypothesis testing and estimation problems, including pharmaceutical, genomics, and economics. To inform readers on the limitations of the method, the book also exhibits counterexamples to the consistency of bootstrap methods. An introduction to R programming provides the needed preparation to work with the numerous exercises and applications presented throughout the book. A related website houses the book's R subroutines, and an extensive listing of references provides resources for further study. Discussing the topic at a remarkably practical and accessible level, An Introduction to Bootstrap Methods with Applications to R is an excellent book for introductory courses on bootstrap and resampling methods at the upper-undergraduate and graduate levels. It also serves as an insightful reference for practitioners working with data in engineering, medicine, and the social sciences who would like to acquire a basic understanding of bootstrap methods.

Table of Contents:
Preface xi Acknowledgments xv List of Tables xvii 1 Introduction 1 1.1 Historical Background 1 1.2 Definition and Relationship to the Delta Method and Other Resampling Methods 3 1.2.1 Jackknife 6 1.2.2 Delta Method 7 1.2.3 Cross-Validation 7 1.2.4 Subsampling 8 1.3 Wide Range of Applications 8 1.4 The Bootstrap and the R Language System 10 1.5 Historical Notes 25 1.6 Exercises 26 References 27 2 Estimation 30 2.1 Estimating Bias 30 2.1.1 Bootstrap Adjustment 30 2.1.2 Error Rate Estimation in Discriminant Analysis 32 2.1.3 Simple Example of Linear Discrimination and Bootstrap Error Rate Estimation 42 2.1.4 Patch Data Example 51 2.2 Estimating Location 53 2.2.1 Estimating a Mean 53 2.2.2 Estimating a Median 54 2.3 Estimating Dispersion 54 2.3.1 Estimating an Estimate’s Standard Error 55 2.3.2 Estimating Interquartile Range 56 2.4 Linear Regression 56 2.4.1 Overview 56 2.4.2 Bootstrapping Residuals 57 2.4.3 Bootstrapping Pairs (Response and Predictor Vector) 58 2.4.4 Heteroscedasticity of Variance: The Wild Bootstrap 58 2.4.5 A Special Class of Linear Regression Models: Multivariable Fractional Polynomials 60 2.5 Nonlinear Regression 60 2.5.1 Examples of Nonlinear Models 61 2.5.2 A Quasi-Optical Experiment 63 2.6 Nonparametric Regression 63 2.6.1 Examples of Nonparametric Regression Models 64 2.6.2 Bootstrap Bagging 66 2.7 Historical Notes 67 2.8 Exercises 69 References 71 3 Confidence Intervals 76 3.1 Subsampling, Typical Value Theorem, and Efron’s Percentile Method 77 3.2 Bootstrap-t 79 3.3 Iterated Bootstrap 83 3.4 Bias-Corrected (BC) Bootstrap 85 3.5 BCa and ABC 85 3.6 Tilted Bootstrap 88 3.7 Variance Estimation with Small Sample Sizes 90 3.8 Historical Notes 94 3.9 Exercises 96 References 98 4 Hypothesis Testing 101 4.1 Relationship to Confidence Intervals 103 4.2 Why Test Hypotheses Differently? 105 4.3 Tendril DX Example 106 4.4 Klingenberg Example: Binary Dose–Response 108 4.5 Historical Notes 109 4.6 Exercises 110 References 111 5 Time Series 113 5.1 Forecasting Methods 113 5.2 Time Domain Models 114 5.3 Can Bootstrapping Improve Prediction Intervals? 115 5.4 Model-Based Methods 118 5.4.1 Bootstrapping Stationary Autoregressive Processes 118 5.4.2 Bootstrapping Explosive Autoregressive Processes 123 5.4.3 Bootstrapping Unstable Autoregressive Processes 123 5.4.4 Bootstrapping Stationary ARMA Processes 123 5.5 Block Bootstrapping for Stationary Time Series 123 5.6 Dependent Wild Bootstrap (DWB) 126 5.7 Frequency-Based Approaches for Stationary Time Series 127 5.8 Sieve Bootstrap 128 5.9 Historical Notes 129 5.10 Exercises 131 References 131 6 Bootstrap Variants 136 6.1 Bayesian Bootstrap 137 6.2 Smoothed Bootstrap 138 6.3 Parametric Bootstrap 139 6.4 Double Bootstrap 139 6.5 The m-Out-of-n Bootstrap 140 6.6 The Wild Bootstrap 141 6.7 Historical Notes 141 6.8 Exercises 142 References 142 7 Chapter Special Topics 144 7.1 Spatial Data 144 7.1.1 Kriging 144 7.1.2 Asymptotics for Spatial Data 147 7.1.3 Block Bootstrap on Regular Grids 148 7.1.4 Block Bootstrap on Irregular Grids 148 7.2 Subset Selection in Regression 148 7.2.1 Gong’s Logistic Regression Example 149 7.2.2 Gunter’s Qualitative Interaction Example 153 7.3 Determining the Number of Distributions in a Mixture 155 7.4 Censored Data 157 7.5 P-Value Adjustment 158 7.5.1 The Westfall–Young Approach 159 7.5.2 Passive Plus Example 159 7.5.3 Consulting Example 160 7.6 Bioequivalence 162 7.6.1 Individual Bioequivalence 162 7.6.2 Population Bioequivalence 165 7.7 Process Capability Indices 165 7.8 Missing Data 172 7.9 Point Processes 174 7.10 Bootstrap to Detect Outliers 176 7.11 Lattice Variables 177 7.12 Covariate Adjustment of Area Under the Curve Estimates for Receiver Operating Characteristic (ROC) Curves 177 7.13 Bootstrapping in SAS 179 7.14 Historical Notes 182 7.15 Exercises 183 References 185 8 When the Bootstrap is Inconsistent and How to Remedy it 190 8.1 Too Small of a Sample Size 191 8.2 Distributions with Infinite Second Moments 191 8.2.1 Introduction 191 8.2.2 Example of Inconsistency 192 8.2.3 Remedies 193 8.3 Estimating Extreme Values 194 8.3.1 Introduction 194 8.3.2 Example of Inconsistency 194 8.3.3 Remedies 194 8.4 Survey Sampling 195 8.4.1 Introduction 195 8.4.2 Example of Inconsistency 195 8.4.3 Remedies 195 8.5 m-Dependent Sequences 196 8.5.1 Introduction 196 8.5.2 Example of Inconsistency When Independence Is Assumed 196 8.5.3 Remedy 197 8.6 Unstable Autoregressive Processes 197 8.6.1 Introduction 197 8.6.2 Example of Inconsistency 197 8.6.3 Remedies 197 8.7 Long-Range Dependence 198 8.7.1 Introduction 198 8.7.2 Example of Inconsistency 198 8.7.3 A Remedy 198 8.8 Bootstrap Diagnostics 199 8.9 Historical Notes 199 8.10 Exercises 201 References 201 Author Index 204 Subject Index 210