Survey Data Harmonization in the Social Sciences

Survey Data Harmonization in the Social Sciences An expansive and incisive overview of the practical uses of harmonization and its implications for data quality and costs In Survey Data Harmonization in the Social Sciences, a team of distinguished social science researchers delivers a comprehensive collection of ex-ante and ex-post harmonization methodologies in the context of specific longitudinal and cross-national survey projects. The book examines how ex-ante and ex-post harmonization work individually and in relation to one another, offering practical guidance on harmonization decisions in the preparation of new data infrastructure for comparative research. Contributions from experts in sociology, political science, demography, economics, health, and medicine are included, all of which give voice to discipline-specific and interdisciplinary views on methodological challenges inherent in harmonization. The authors offer perspectives from Europe and the United States, as well as Africa, the latter of which provides insights rarely featured in survey research methodology handbooks. Readers will also find: A thorough introduction to approaches and concepts for survey data harmonization, as well as the effects of data harmonization on the overall survey research process Comprehensive explorations of ex-ante harmonization of survey instruments and non-survey data Practical discussions of ex-post harmonization of national social surveys, census and time use data, including explorations of survey data recycling A detailed overview of statistical issues linked to the use of harmonized survey data Perfect for upper undergraduate and graduate researchers who specialize in survey methodology, Survey Data Harmonization in the Social Sciences will also earn a place in the libraries of survey practitioners who engage in international research.

Table of Contents:
Preface and Acknowledgments xv About the Editors xvii About the Contributors xviii 1 Objectives and Challenges of Survey Data Harmonization 1 Kazimierz M. Slomczynski, Irina Tomescu-Dubrow, J. Craig Jenkins, and Christof Wolf 1.1 Introduction 1 1.2 What is the Harmonization of Survey Data? 2 1.2.1 Ex-ante, Input and Output, Survey Harmonization 3 1.3 Why Harmonize Social Survey Data? 5 1.3.1 Comparison and Equivalence 6 1.4 Harmonizing Survey Data Across and Within Countries 7 1.4.1 Harmonizing Across Countries 7 1.4.2 Harmonizing Within the Country 8 1.5 Sources of Knowledge for Survey Data Harmonization 8 1.6 Challenges to Survey Harmonization 9 1.6.1 Population Representation (Sampling Design) 10 1.6.2 Instruments and Their Adaptation (Including Translation) 10 1.6.3 Preparation for Interviewing (Including Pretesting) 11 1.6.4 Fieldwork (Including Modes of Interviewing) 11 1.6.5 Data Preparation (Including Building Data Files) 12 1.6.6 Data Processing, Quality Controls, and Adjustments 12 1.6.7 Data Dissemination 13 1.7 Survey Harmonization and Standardization Processes 13 1.8 Quality of the Input and the End-product of Survey Harmonization 14 1.9 Relevance of Harmonization Methodology to the FAIR Data Principles 15 1.10 Ethical and Legal Issues 15 1.11 How to Read this Volume? 16 References 17 2 The Effects of Data Harmonization on the Survey Research Process 21 Ranjit K. Singh, Arnim Bleier, and Peter Granda 2.1 Introduction 21 2.2 Part 1: Harmonization: Origins and Relation to Standardization 22 2.2.1 Early Conceptions of Standardization and Harmonization 22 2.2.2 Foundational Work of International Survey Programs 23 2.2.3 The Growing Impact of Data Harmonization 23 2.3 Part 2: Stakeholders and Division of Labor 25 2.3.1 Stakeholders 26 2.3.1.1 International Actors and Funding Agencies 26 2.3.1.2 Data Producers 26 2.3.1.3 Archives 27 2.3.1.4 Data Users 27 2.3.2 Toward an Integrative View on Harmonization 28 2.3.2.1 Harmonization Cost 29 2.3.2.2 Harmonization Quality 29 2.3.2.3 Harmonization Fit 30 2.3.2.4 Moving Forward 30 2.4 Part 3: New Data Types, New Challenges 31 2.4.1 Designed Data and Organic Data 31 2.4.2 Stakeholders in the Collection of Organic Data 32 2.4.2.1 Producers 32 2.4.2.2 Archives 32 2.4.2.3 Users 33 2.4.2.4 Harmonization of Organic Data 33 2.5 Conclusion 33 References 35 Part I Ex-ante harmonization of survey instruments and non-survey data 39 3 Harmonization in the World Values Survey 41 Kseniya Kizilova, Jaime Diez-Medrano, Christian Welzel, and Christian Haerpfer 3.1 Introduction 41 3.2 Applied Harmonization Methods 42 3.3 Documentation and Quality Assurance 48 3.4 Challenges to Harmonization 49 3.5 Software Tools 51 3.6 Recommendations 52 References 54 4 Harmonization in the Afrobarometer 57 Carolyn Logan, Robert Mattes, and Francis Kibirige 4.1 Introduction 57 4.2 Core Principles 58 4.3 Applied Harmonization Methods 60 4.3.1 Sampling 60 4.3.2 Training 61 4.3.3 Fieldwork and Data Collection 62 4.3.4 Questionnaire 62 4.3.5 Translation 64 4.3.6 Data Management 65 4.3.7 Documentation 65 4.4 Harmonization and Country Selection 66 4.5 Software Tools and Harmonization 66 4.6 Challenges to Harmonization 67 4.6.1 Local Knowledge, Flexibility/Adaptability, and the “Dictatorship of Harmonization” 68 4.6.2 The Quality-Cost Trade-off and Implications for Harmonization 68 4.6.3 Final Challenge: “Events” 69 4.7 Recommendations 70 References 71 5 Harmonization in the National Longitudinal Surveys of Youth (NLSY) 73 Elizabeth Cooksey, Rosella Gardecki, Carole Lunney, and Amanda Roose 5.1 Introduction 73 5.2 Cross-Cohort Design 75 5.3 Applied Harmonization 76 5.4 Challenges to Harmonization 80 5.5 Documentation and Quality Assurance 82 5.6 Software Tools 84 5.7 Recommendations and Some Concluding Thoughts 86 References 87 6 Harmonization in the Comparative Study of Electoral Systems (CSES) Projects 89 Stephen Quinlan, Christian Schimpf, Katharina Blinzler, and Slaven Zivkovic 6.1 Introducing the CSES 89 6.2 Harmonization Principles and Technical Infrastructure 91 6.3 Ex-ante Input Harmonization 91 6.3.1 Module Questionnaire 92 6.3.2 Macro Data 94 6.4 Ex-ante Output Harmonization 97 6.4.1 Demographic Variables in CSES Modules 97 6.4.2 Harmonizing Party Data in Modules 98 6.4.3 Derivative Variables 99 6.5 Exploring Interplay Between Ex-ante and Ex-post Harmonization 101 6.5.1 Demographic Variables in CSES IMD 101 6.5.2 Harmonizing Party Data in CSES IMD 102 6.6 Taking Stock and New Frontiers in Harmonization 104 References 105 7 Harmonization in the East Asian Social Survey 107 Noriko Iwai, Tetsuo Mo, Jibum Kim, Chyi-In Wu, and Weidong Wang 7.1 Introduction 107 7.2 Characteristics of the EASS and its Harmonization Process 108 7.2.1 Outline of the East Asian Social Survey 108 7.2.2 Harmonization Process of the EASS 111 7.2.2.1 Establishing the Module Theme 111 7.2.2.2 Selecting Subtopics and Questions 112 7.2.2.3 Harmonization of Standard Background Variables 113 7.2.2.4 Harmonization of Answer Choices and Scales 114 7.2.2.5 Translation of Questions and Answer Choices 115 7.3 Documentation and Quality Assurance 115 7.3.1 Five Steps to Harmonize the EASS Integrated Data 115 7.3.2 Documentation of the EASS Integrated Data 117 7.4 Challenges to Harmonization 118 7.4.1 How to Translate “Fair” and Restriction by Copyright 118 7.4.2 Difficulty in Synchronizing the Data Collection Phase 121 7.5 Software Tools 122 7.6 Recommendations 122 Acknowledgment 123 References 123 8 Ex-ante Harmonization of Official Statistics in Africa (SHaSA) 125 Dossina Yeo Abbreviations 125 8.1 Introduction 127 8.2 Applied Harmonization Methods 128 8.2.1 Examples of Ex-ante Harmonization Methods: The Cases of GPS Data and CRVS 131 8.2.1.1 Governance, Peace and Security (GPS) Statistics Initiative 131 8.2.1.2 Development of Civil Registration and Vital Statistics (CRVS) 132 8.2.2 Examples of Ex-post Harmonization: The Cases of Labor Statistics, ATSY, ASY and KeyStats, and ICP-Africa Program 132 8.3 Quality Assurance Framework 134 8.4 Challenges to Statistical Harmonization in Africa 136 8.4.1 Challenges to the Implementation of NSDS 137 8.4.2 Challenges with Ex-ante Harmonization: Examples of GPS and ICP Initiatives 138 8.4.3 Challenges with Ex-post Harmonization: Examples of KeyStats and ATSY 139 8.5 Common Software Tools Used 139 8.6 Conclusion and Recommendations 140 References 142 Part II Ex-post harmonization of national social surveys 145 9 Harmonization for Cross-National Secondary Analysis: Survey Data Recycling 147 Irina Tomescu-Dubrow, Kazimierz M. Slomczynski, Ilona Wysmulek, Przemek Powałko, Olga Li, Yamei Tu, Marcin Slarzynski, Marcin W. Zielinski, and Denys Lavryk 9.1 Introduction 147 9.2 Harmonization Methods in the SDR Project 149 9.2.1 Building the Harmonized SDR2 Database 150 9.3 Documentation and Quality Assurance 155 9.4 Challenges to Harmonization 156 9.5 Software Tools of the SDR Project 161 9.5.1 The SDR Portal 161 9.5.2 The SDR2 COTTON FILE 162 9.6 Recommendations 162 9.6.1 Recommendations for Researchers Interested in Harmonizing Survey Data Ex-Post 162 9.6.2 Recommendations for SDR2 Users 163 Acknowledgments 164 References 164 9.A Data Quality Indicators in SDR2 166 10 Harmonization of Panel Surveys: The Cross-National Equivalent File 169 Dean R. Lillard 10.1 Introduction 169 10.2 Applied Harmonization Methods 170 10.2.1 CNEF Country Data Sources, Current and Planned 176 10.3 Current CNEF Partners 176 10.3.1 The HILDA Survey 176 10.3.2 The SLID 176 10.3.3 The CFPS 177 10.3.4 The SOEP 177 10.3.4.1 The BHPS 177 10.3.4.2 Understanding Society, UKHLS 178 10.3.5 The ITA.LI 178 10.3.6 The JHPS 178 10.3.7 The RLMS-HSE 178 10.3.8 The KLIPS 179 10.3.9 The Swedish Pseudo-Panel 179 10.3.10 The SHP 179 10.3.11 The PSID 179 10.4 Planned CNEF Partners 180 10.4.1 The ASEP 180 10.4.2 LISA 180 10.4.3 The ILS 180 10.4.4 The MxFLS 180 10.4.5 The NIDS 181 10.4.6 The PSFD 181 10.5 Documentation and Quality Assurance 181 10.6 Challenges to Harmonization 183 10.7 Recommendations for Researchers Interested in Harmonizing Panel Survey Data 185 10.8 Conclusion 186 References 187 11 Harmonization of Survey Data from UK Longitudinal Studies: CLOSER 189 Dara O’Neill and Rebecca Hardy 11.1 Introduction 189 11.2 Applied Harmonization Methods 191 11.2.1 Occupational Social Class 191 11.2.2 Body Size/Anthropometric Data 193 11.2.3 Mental Health 194 11.2.4 Harmonization Methods: Divergence and Convergence 195 11.3 Documentation and Quality Assurance 196 11.4 Challenges to Harmonization 198 11.5 Software Tools 199 11.6 Recommendations 200 Acknowledgments 202 References 202 12 Harmonization of Census Data: IPUMS – International 207 Steven Ruggles, Lara Cleveland, and Matthew Sobek 12.1 Introduction 207 12.2 Project History 208 12.2.1 Evolution of the Web Dissemination System 210 12.3 Applied Harmonization Methods 210 12.4 Documentation and Quality Assurance 215 12.5 Challenges to Harmonization 217 12.6 Software Tools 221 12.6.1 Metadata Tools 221 12.6.2 Data Reformatting 221 12.6.3 Data Harmonization 221 12.6.4 Dissemination System 222 12.7 Team Organization and Project Management 222 12.8 Lessons and Recommendations 223 References 225 Part III Domain-driven ex-post harmonization 227 13 Maelstrom Research Approaches to Retrospective Harmonization of Cohort Data for Epidemiological Research 229 Tina W. Wey and Isabel Fortier 13.1 Introduction 229 13.2 Applied Harmonization Methods 230 13.2.1 Implementing the Project 233 13.2.1.1 Initiating Activities and Organizing the Operational Framework 233 13.2.1.2 Assembling Study Information and Selecting Final Participating Studies (Guidelines Step 1) 234 13.2.1.3 Defining Target Variables to be Harmonized (the DataSchema) and Evaluating Harmonization Potential across Studies (Guidelines Step 2) 235 13.2.2 Producing the Harmonized Datasets 236 13.2.2.1 Processing Data (Guidelines Step 3a) 236 13.2.2.2 Processing Study-Specific Data to Generate Harmonized Datasets (Guidelines Step 3b) 237 13.3 Documentation and Quality Assurance 238 13.4 Challenges to Harmonization 240 13.5 Software Tools 241 13.6 Recommendations 243 Acknowledgments 244 References 245 14 Harmonizing and Synthesizing Partnership Histories from Different German Survey Infrastructures 249 Bernd Weiß, Sonja Schulz, Lisa Schmid, Sebastian Sterl, and Anna-Carolina Haensch 14.1 Introduction 249 14.2 Applied Harmonization Methods 250 14.2.1 Data Search Strategy and Data Access 250 14.2.2 Processing and Harmonizing Data 253 14.2.2.1 Harmonizing Partnership Biography Data 253 14.2.2.2 Harmonizing Additional Variables on Respondents’ or Couples’ Characteristics 254 14.3 Documentation and Quality Assurance 255 14.3.1 Documentation 255 14.3.2 Quality Assurance 256 14.3.2.1 Process-Related Quality Assurance 256 14.3.2.2 Benchmarking the Harmonized HaSpaD Data Set with Official Statistics 256 14.4 Challenges to Harmonization 258 14.4.1 Analyzing Harmonized Complex Survey Data 258 14.4.2 Sporadically and Systematically Missing Data 259 14.5 Software Tools 260 14.6 Recommendations 262 14.6.1 Harmonizing Biographical Data 262 14.6.1.1 Methodological Recommendations 262 14.6.1.2 Procedural Recommendations 263 14.6.1.3 Technical Recommendations 263 14.6.2 Getting Started with the Cumulative HaSpaD Data Set 263 Acknowledgments 264 References 264 15 Harmonization and Quality Assurance of Income and Wealth Data: The Case of LIS 269 Jörg Neugschwender, Teresa Munzi, and Piotr R. Paradowski 15.1 Introduction 269 15.2 Applied Harmonization Methods 271 15.3 Documentation and Quality Assurance 275 15.3.1 Quality Assurance 275 Selection of Source Datasets 276 Harmonization 276 Validation – “Green Light” Check 276 15.3.2 Documentation 278 15.4 Challenges to Harmonization 278 15.5 Software Tools 281 15.6 Conclusion 282 References 283 16 Ex-Post Harmonization of Time Use Data: Current Practices and Challenges in the Field 285 Ewa Jarosz, Sarah Flood, and Margarita Vega-Rapun 16.1 Introduction 285 16.2 Applied Harmonization Methods 289 16.2.1 Harmonizing the Matrix of the Diary 289 16.2.2 Variable Harmonization 291 16.2.3 Other Variables 293 16.2.4 Other Types of Time Use Data 294 16.3 Documentation and Quality Assurance 294 16.3.1 Documentation 294 16.3.2 Quality Checks 296 16.4 Challenges to Harmonization 297 16.5 Software Tools 300 16.6 Recommendations 301 References 302 Part IV Further Issues: Dealing with Methodological Issues in Harmonized Survey Data 305 17 Assessing and Improving the Comparability of Latent Construct Measurements in Ex-Post Harmonization 307 Ranjit K. Singh and Markus Quandt 17.1 Introduction 307 17.2 Measurement and Reality 307 17.3 Construct Match 308 17.3.1 Consequences of a Mismatch 309 17.3.2 Assessment 309 17.3.2.1 Qualitative Research Methods 309 17.3.2.2 Construct and Criterion Validity 309 17.3.2.3 Techniques for Multi-Item Instruments 310 17.3.2.4 Improving Construct Comparability 311 17.4 Reliability Differences 311 17.4.1 Consequences of Reliability Differences 311 17.4.2 Assessment 312 17.4.3 Improving Reliability Comparability 312 17.5 Units of Measurement 312 17.5.1 Consequences of Unit Differences 313 17.5.2 Improving Unit Comparability 313 17.5.3 Controlling for Instrument Characteristics 314 17.5.4 Harmonizing Units Based on Repeated Measurements 315 17.5.5 Harmonizing Units Based on Measurements Obtained from the Same Population 315 17.6 Cross-Cultural Comparability 316 17.6.1 Construct Match 316 17.6.1.1 Translation and Cognitive Probing 317 17.6.2 Reliability 317 17.6.3 Units of Measurement 318 17.6.3.1 Harmonizing Units of Localized Versions of the Same Instrument 318 17.6.3.2 Harmonizing Units Across Cultures and Instruments 318 17.6.4 Cross-Cultural Comparability of Multi-Item Instruments 318 17.7 Discussion and Outlook 319 References 320 18 Comparability and Measurement Invariance 323 Artur Pokropek 18.1 Latent Variable Framework for Testing and Accounting for Measurement Non-Invariance 324 18.2 Approaches to Empirical Assessment of Measurement Equivalence 325 18.2.1 Classical Invariance Analysis (MG-CFA) 326 18.2.2 Partial Invariance (MG-CFA) 327 18.2.3 Approximate Invariance 327 18.2.4 Approximate Partial Invariance (Alignment, BSEM Alignment, Partial BSEM) 328 18.3 Beyond Multiple Indicators 329 18.4 Conclusions 329 References 330 19 On the Creation, Documentation, and Sensible Use of Weights in the Context of Comparative Surveys 333 Dominique Joye, Marlène Sapin, and Christof Wolf 19.1 Introduction 333 19.2 Design Weights 335 19.2.1 What to do? 336 19.3 Post-stratification Weights 337 19.3.1 What Should be Done? 340 19.4 Population Weights 341 19.4.1 What Should be Done? 342 19.5 Conclusion 342 References 344 20 On Using Harmonized Data in Statistical Analysis: Notes of Caution 347 Claire Durand 20.1 Introduction 347 20.2 Challenges in the Combination of Data Sets 347 20.2.1 A First Principle: A No Censorship Inclusive Approach 348 20.2.2 A Second Principle: Using Multilevel Analysis and Introducing a Measurement Level 349 20.2.3 A Third Principle: Assessing the Equivalence of Survey Projects 351 20.3 Challenges in the Analysis of Combined Data Sets 353 20.3.1 Dealing with Time 354 20.3.2 Dealing with Missing Values 358 20.3.2.1 Missing Values at the Respondent and Measurement Level 358 20.3.2.2 Missing Values at the Survey Level 359 20.3.3 Dealing with Weights 361 20.4 Recommendations 362 References 363 21 On the Future of Survey Data Harmonization 367 Kazimierz M. Slomczynski, Christof Wolf, Irina Tomescu-Dubrow, and J. Craig Jenkins 21.1 What We Have Learned from Contributions on Survey Data Harmonization in this Volume 368 21.2 New Opportunities and Challenges 370 21.2.1 Reorientation of Survey Research in the Era of New Technology 370 21.2.2 Advances in Technical Aspects of Data Management 370 21.2.3 Harmonizing Survey Data with Other Types of Data 371 21.3 Developing a New Methodology of Harmonizing Non-Survey Data 372 21.3.1 Emerging Legal and Ethical Issues 372 21.4 Globalization of Science and Harmonizing Scientific Practice 373 References 373 Index 377