Introduction to Meta–Analysis

This book provides a clear and thorough introduction to meta-analysis, the process of synthesizing data from a series of separate studies. Meta-analysis has become a critically important tool in fields as diverse as medicine, pharmacology, epidemiology, education, psychology, business, and ecology. Introduction to Meta-Analysis:* Outlines the role of meta-analysis in the research process* Shows how to compute effects sizes and treatment effects* Explains the fixed-effect and random-effects models for synthesizing data* Demonstrates how to assess and interpret variation in effect size across studies* Clarifies concepts using text and figures, followed by formulas and examples* Explains how to avoid common mistakes in meta-analysis* Discusses controversies in meta-analysis* Features a web site with additional material and exercises A superb combination of lucid prose and informative graphics, written by four of the world's leading experts on all aspects of meta-analysis. Borenstein, Hedges, Higgins, and Rothstein provide a refreshing departure from cookbook approaches with their clear explanations of the what and why of meta-analysis. The book is ideal as a course textbook or for self-study. My students, who used pre-publication versions of some of the chapters, raved about the clarity of the explanations and examples. David Rindskopf, Distinguished Professor of Educational Psychology, City University of New York, Graduate School and University Center, & Editor of the Journal of Educational and Behavioral Statistics. The approach taken by Introduction to Meta-analysis is intended to be primarily conceptual, and it is amazingly successful at achieving that goal. The reader can comfortably skip the formulas and still understand their application and underlying motivation. For the more statistically sophisticated reader, the relevant formulas and worked examples provide a superb practical guide to performing a meta-analysis. The book provides an eclectic mix of examples from education, social science, biomedical studies, and even ecology. For anyone considering leading a course in meta-analysis, or pursuing self-directed study, Introduction to Meta-analysis would be a clear first choice. Jesse A. Berlin, ScD Introduction to Meta-Analysis is an excellent resource for novices and experts alike. The book provides a clear and comprehensive presentation of all basic and most advanced approaches to meta-analysis. This book will be referenced for decades. Michael A. McDaniel, Professor of Human Resources and Organizational Behavior, Virginia Commonwealth University

Table of Contents:
List of Figures List of Tables Acknowledgements Preface PART 1: INTRODUCTION 1 HOW A META-ANALYSIS WORKS Introduction Individual studies The summary effect Heterogeneity of effect sizes Summary points 2 WHY PERFORM A META-ANALYSIS Introduction The SKIV meta-analysis Statistical significance Clinical importance of the effect Consistency of effects Summary points PART 2: EFFECT SIZE AND PRECISION 3 OVERVIEW Treatment effects and effect sizes Parameters and estimates Outline 4 EFFECT SIZES BASED ON MEANS Introduction Raw (unstandardized) mean difference D Standardized mean difference, D and G Response ratios Summary points 5 EFFECT SIZES BASED ON BINARY DATA (2x2 TABLES) Introduction Risk ratio Odds ratio Risk difference Choosing an effect size index Summary points 6 EFFECT SIZES BASED ON CORRELATIONS Introduction Computing R Other approaches Summary points 7 CONVERTING AMONG EFFECT SIZES Introduction Converting from the log odds ratio to D Converting from D to the log odds ratio Converting from R to D Converting from D to R Summary points 8 FACTORS THAT AFFECT PRECISION Introduction Factors that affect precision Sample size Study design Summary points 9 CONCLUDING REMARKS Further reading PART 3: FIXED-EFFECT VERSUS RANDOM-EFFECTS MODELS 10 OVERVIEW Introduction Nomenclature 11 FIXED-EFFECT MODEL Introduction The true effect size Impact of sampling error Performing a fixed-effect meta-analysis Summary points 12 RANDOM-EFFECTS MODEL Introduction The true effect sizes Impact of sampling error Performing a random-effects meta-analysis Summary points 13 FIXED EFFECT VERSUS RANDOM-EFFECTS MODELS Introduction Definition of a summary effect Estimating the summary effect Extreme effect size in large study Confidence interval The null hypothesis Which model should we use? Model should not be based on the test for heterogeneity Concluding remarks Summary points 14 WORKED EXAMPLES (PART 1) Introduction Worked example for continuous data (Part 1) Worked example for binary data (Part 1) Worked example for correlational data (Part 1) Summary points PART 4: HETEROGENEITY 15 OVERVIEW Introduction 16 IDENTIFYING AND QUANTIFYING HETEROGENEITY Introduction Isolating the variation in true effects Computing Q Estimating tau-squared The I 2 statistic Comparing the measures of heterogeneity Confidence intervals for T 2 Confidence intervals (or uncertainty intervals) for I 2 Summary points 17 PREDICTION INTERVALS Introduction Prediction intervals in primary studies Prediction intervals in meta-analysis Confidence intervals and prediction intervals Comparing the confidence interval with the prediction interval Summary points 18 WORKED EXAMPLES (PART 2) Introduction Worked example for continuous data (Part 2) Worked example for binary data (Part 2) Worked example for correlational data (Part 2) Summary points 19 SUBGROUP ANALYSES Introduction Fixed-effect model within subgroups Computational models Random effects with separate estimates of T 2 Random effects with pooled estimate of T 2 The proportion of variance explained Mixed-effect model Obtaining an overall effect in the presence of subgroups Summary points 20 META-REGRESSION Introduction Fixed-effect model Fixed or random effects for unexplained heterogeneity Random-effects model Statistical power for regression Summary points 21 NOTES ON SUBGROUP ANALYSES AND META-REGRESSION Introduction Computational model Multiple comparisons Software Analysis of subgroups and regression are observational Statistical power for subgroup analyses and meta-regression Summary points PART 5: COMPLEX DATA STRUCTURES 22 OVERVIEW 23 INDEPENDENT SUBGROUPS WITHIN A STUDY Introduction Combining across subgroups Comparing subgroups Summary points 24 MULTIPLE OUTCOMES OR TIME POINTS WITHIN A STUDY Introduction Combining across outcomes or time-points Comparing outcomes or time-points within a study Summary points 25 MULTIPLE COMPARISONS WITHIN A STUDY Introduction Combining across multiple comparisons within a study Differences between treatments Summary points 26 NOTES ON COMPLEX DATA STRUCTURES Introduction Combined effect Differences in effect PART 6: OTHER ISSUES 27 OVERVIEW 28 VOTE COUNTING - A NEW NAME FOR AN OLD PROBLEM Introduction Why vote counting is wrong Vote-counting is a pervasive problem Summary points 29 POWER ANALYSIS FOR META-ANALYSIS Introduction A conceptual approach In context When to use power analysis Planning for precision rather than for power Power analysis in primary studies Power analysis for meta-analysis Power analysis for a test of homogeneity Summary points 30 PUBLICATION BIAS Introduction The problem of missing studies Methods for addressing bias Illustrative example The model Getting a sense of the data Is the entire effect an artifact of bias How much of an impact might the bias have? Summary of the findings for the illustrative example Small study effects Concluding remarks Summary points PART 7: ISSUES RELATED TO EFFECT SIZE 31 OVERVIEW 32 EFFECT SIZES RATHER THAN P -VALUES Introduction Relationship between p-values and effect sizes The distinction is important The p-value is often misinterpreted Narrative reviews vs. meta-analyses Summary points 33 SIMPSON'S PARADOX Introduction Circumcision and risk of HIV infection An example of the paradox Summary points 34 GENERALITY OF THE BASIC INVERSE-VARIANCE METHOD Introduction Other effect sizes Other methods for estimating effect sizes Individual participant data meta-analyses Bayesian approaches Summary points PART 8: FURTHER METHODS 35 OVERVIEW 36 META-ANALYSIS METHODS BASED ON DIRECTION AND P -VALUES Introduction Vote counting The sign test Combining p-values Summary points 37 FURTHER METHODS FOR DICHOTOMOUS DATA Introduction Mantel-Haenszel method One-step (Peto) formula for odds ratio Summary points 38 PSYCHOMETRIC META-ANALYSIS Introduction The attenuating effects of artifacts Meta-analysis methods Example of psychometric meta-analysis Comparison of artifact correction with meta-regression Sources of information about artifact values How heterogeneity is assessed Reporting in psychometric meta-analysis Concluding remarks Summary points PART 9: META-ANALYSIS IN CONTEXT 39 OVERVIEW 40 WHEN DOES IT MAKE SENSE TO PERFORM A META-ANALYSIS? Introduction Are the studies similar enough to combine? Can I combine studies with different designs? How many studies are enough to carry out a meta-analysis? Summary points 41 REPORTING THE RESULTS OF A META-ANALYSIS Introduction The computational model Forest plots Sensitivity analysis Summary points 42 CUMULATIVE META-ANALYSIS Introduction Why perform a cumulative meta-analysis? Summary points 43 CRITICISMS OF META-ANALYSIS Introduction One number cannot summarize a research field The file drawer problem invalidates meta-analysis Mixing apples and oranges Garbage in, garbage out Important studies are ignored Meta-analysis can disagree with randomized trials Meta-analyses are performed poorly Is a narrative review better? Concluding remarks Summary points PART 10: RESOURCES AND SOFTWARE 44 SOFTWARE Introduction Three examples of meta-analysis software The software Comprehensive meta-analysis (CMA) 2.0 Revman 5.0 StataTM macros with Stata 10.0 Summary points 45 BOOKS, WEB SITES AND PROFESSIONAL ORGANIZATIONS Books on systematic review methods Books on meta-analysis Web sites INDEX