Survey Data Harmonization in the Social Sciences An expansive and incisive overview of the practical uses of harmonization and its implications for data quality and costs
In Survey Data Harmonization in the Social Sciences, a team of distinguished social science researchers delivers a comprehensive collection of ex-ante and ex-post harmonization methodologies in the context of specific longitudinal and cross-national survey projects. The book examines how ex-ante and ex-post harmonization work individually and in relation to one another, offering practical guidance on harmonization decisions in the preparation of new data infrastructure for comparative research.
Contributions from experts in sociology, political science, demography, economics, health, and medicine are included, all of which give voice to discipline-specific and interdisciplinary views on methodological challenges inherent in harmonization. The authors offer perspectives from Europe and the United States, as well as Africa, the latter of which provides insights rarely featured in survey research methodology handbooks.
Readers will also find:
- A thorough introduction to approaches and concepts for survey data harmonization, as well as the effects of data harmonization on the overall survey research process
- Comprehensive explorations of ex-ante harmonization of survey instruments and non-survey data
- Practical discussions of ex-post harmonization of national social surveys, census and time use data, including explorations of survey data recycling
- A detailed overview of statistical issues linked to the use of harmonized survey data
Perfect for upper undergraduate and graduate researchers who specialize in survey methodology, Survey Data Harmonization in the Social Sciences will also earn a place in the libraries of survey practitioners who engage in international research.
Table of Contents:
Preface and Acknowledgments xv
About the Editors xvii
About the Contributors xviii
1 Objectives and Challenges of Survey Data Harmonization 1
Kazimierz M. Slomczynski, Irina Tomescu-Dubrow, J. Craig Jenkins, and Christof Wolf
1.1 Introduction 1
1.2 What is the Harmonization of Survey Data? 2
1.2.1 Ex-ante, Input and Output, Survey Harmonization 3
1.3 Why Harmonize Social Survey Data? 5
1.3.1 Comparison and Equivalence 6
1.4 Harmonizing Survey Data Across and Within Countries 7
1.4.1 Harmonizing Across Countries 7
1.4.2 Harmonizing Within the Country 8
1.5 Sources of Knowledge for Survey Data Harmonization 8
1.6 Challenges to Survey Harmonization 9
1.6.1 Population Representation (Sampling Design) 10
1.6.2 Instruments and Their Adaptation (Including Translation) 10
1.6.3 Preparation for Interviewing (Including Pretesting) 11
1.6.4 Fieldwork (Including Modes of Interviewing) 11
1.6.5 Data Preparation (Including Building Data Files) 12
1.6.6 Data Processing, Quality Controls, and Adjustments 12
1.6.7 Data Dissemination 13
1.7 Survey Harmonization and Standardization Processes 13
1.8 Quality of the Input and the End-product of Survey Harmonization 14
1.9 Relevance of Harmonization Methodology to the FAIR Data Principles 15
1.10 Ethical and Legal Issues 15
1.11 How to Read this Volume? 16
References 17
2 The Effects of Data Harmonization on the Survey Research Process 21
Ranjit K. Singh, Arnim Bleier, and Peter Granda
2.1 Introduction 21
2.2 Part 1: Harmonization: Origins and Relation to Standardization 22
2.2.1 Early Conceptions of Standardization and Harmonization 22
2.2.2 Foundational Work of International Survey Programs 23
2.2.3 The Growing Impact of Data Harmonization 23
2.3 Part 2: Stakeholders and Division of Labor 25
2.3.1 Stakeholders 26
2.3.1.1 International Actors and Funding Agencies 26
2.3.1.2 Data Producers 26
2.3.1.3 Archives 27
2.3.1.4 Data Users 27
2.3.2 Toward an Integrative View on Harmonization 28
2.3.2.1 Harmonization Cost 29
2.3.2.2 Harmonization Quality 29
2.3.2.3 Harmonization Fit 30
2.3.2.4 Moving Forward 30
2.4 Part 3: New Data Types, New Challenges 31
2.4.1 Designed Data and Organic Data 31
2.4.2 Stakeholders in the Collection of Organic Data 32
2.4.2.1 Producers 32
2.4.2.2 Archives 32
2.4.2.3 Users 33
2.4.2.4 Harmonization of Organic Data 33
2.5 Conclusion 33
References 35
Part I Ex-ante harmonization of survey instruments and non-survey data 39
3 Harmonization in the World Values Survey 41
Kseniya Kizilova, Jaime Diez-Medrano, Christian Welzel, and Christian Haerpfer
3.1 Introduction 41
3.2 Applied Harmonization Methods 42
3.3 Documentation and Quality Assurance 48
3.4 Challenges to Harmonization 49
3.5 Software Tools 51
3.6 Recommendations 52
References 54
4 Harmonization in the Afrobarometer 57
Carolyn Logan, Robert Mattes, and Francis Kibirige
4.1 Introduction 57
4.2 Core Principles 58
4.3 Applied Harmonization Methods 60
4.3.1 Sampling 60
4.3.2 Training 61
4.3.3 Fieldwork and Data Collection 62
4.3.4 Questionnaire 62
4.3.5 Translation 64
4.3.6 Data Management 65
4.3.7 Documentation 65
4.4 Harmonization and Country Selection 66
4.5 Software Tools and Harmonization 66
4.6 Challenges to Harmonization 67
4.6.1 Local Knowledge, Flexibility/Adaptability, and the “Dictatorship of Harmonization” 68
4.6.2 The Quality-Cost Trade-off and Implications for Harmonization 68
4.6.3 Final Challenge: “Events” 69
4.7 Recommendations 70
References 71
5 Harmonization in the National Longitudinal Surveys of Youth (NLSY) 73
Elizabeth Cooksey, Rosella Gardecki, Carole Lunney, and Amanda Roose
5.1 Introduction 73
5.2 Cross-Cohort Design 75
5.3 Applied Harmonization 76
5.4 Challenges to Harmonization 80
5.5 Documentation and Quality Assurance 82
5.6 Software Tools 84
5.7 Recommendations and Some Concluding Thoughts 86
References 87
6 Harmonization in the Comparative Study of Electoral Systems (CSES) Projects 89
Stephen Quinlan, Christian Schimpf, Katharina Blinzler, and Slaven Zivkovic
6.1 Introducing the CSES 89
6.2 Harmonization Principles and Technical Infrastructure 91
6.3 Ex-ante Input Harmonization 91
6.3.1 Module Questionnaire 92
6.3.2 Macro Data 94
6.4 Ex-ante Output Harmonization 97
6.4.1 Demographic Variables in CSES Modules 97
6.4.2 Harmonizing Party Data in Modules 98
6.4.3 Derivative Variables 99
6.5 Exploring Interplay Between Ex-ante and Ex-post Harmonization 101
6.5.1 Demographic Variables in CSES IMD 101
6.5.2 Harmonizing Party Data in CSES IMD 102
6.6 Taking Stock and New Frontiers in Harmonization 104
References 105
7 Harmonization in the East Asian Social Survey 107
Noriko Iwai, Tetsuo Mo, Jibum Kim, Chyi-In Wu, and Weidong Wang
7.1 Introduction 107
7.2 Characteristics of the EASS and its Harmonization Process 108
7.2.1 Outline of the East Asian Social Survey 108
7.2.2 Harmonization Process of the EASS 111
7.2.2.1 Establishing the Module Theme 111
7.2.2.2 Selecting Subtopics and Questions 112
7.2.2.3 Harmonization of Standard Background Variables 113
7.2.2.4 Harmonization of Answer Choices and Scales 114
7.2.2.5 Translation of Questions and Answer Choices 115
7.3 Documentation and Quality Assurance 115
7.3.1 Five Steps to Harmonize the EASS Integrated Data 115
7.3.2 Documentation of the EASS Integrated Data 117
7.4 Challenges to Harmonization 118
7.4.1 How to Translate “Fair” and Restriction by Copyright 118
7.4.2 Difficulty in Synchronizing the Data Collection Phase 121
7.5 Software Tools 122
7.6 Recommendations 122
Acknowledgment 123
References 123
8 Ex-ante Harmonization of Official Statistics in Africa (SHaSA) 125
Dossina Yeo
Abbreviations 125
8.1 Introduction 127
8.2 Applied Harmonization Methods 128
8.2.1 Examples of Ex-ante Harmonization Methods: The Cases of GPS Data and CRVS 131
8.2.1.1 Governance, Peace and Security (GPS) Statistics Initiative 131
8.2.1.2 Development of Civil Registration and Vital Statistics (CRVS) 132
8.2.2 Examples of Ex-post Harmonization: The Cases of Labor Statistics, ATSY, ASY and KeyStats, and ICP-Africa Program 132
8.3 Quality Assurance Framework 134
8.4 Challenges to Statistical Harmonization in Africa 136
8.4.1 Challenges to the Implementation of NSDS 137
8.4.2 Challenges with Ex-ante Harmonization: Examples of GPS and ICP Initiatives 138
8.4.3 Challenges with Ex-post Harmonization: Examples of KeyStats and ATSY 139
8.5 Common Software Tools Used 139
8.6 Conclusion and Recommendations 140
References 142
Part II Ex-post harmonization of national social surveys 145
9 Harmonization for Cross-National Secondary Analysis: Survey Data Recycling 147
Irina Tomescu-Dubrow, Kazimierz M. Slomczynski, Ilona Wysmulek, Przemek Powałko, Olga Li, Yamei Tu, Marcin Slarzynski, Marcin W. Zielinski, and Denys Lavryk
9.1 Introduction 147
9.2 Harmonization Methods in the SDR Project 149
9.2.1 Building the Harmonized SDR2 Database 150
9.3 Documentation and Quality Assurance 155
9.4 Challenges to Harmonization 156
9.5 Software Tools of the SDR Project 161
9.5.1 The SDR Portal 161
9.5.2 The SDR2 COTTON FILE 162
9.6 Recommendations 162
9.6.1 Recommendations for Researchers Interested in Harmonizing Survey Data Ex-Post 162
9.6.2 Recommendations for SDR2 Users 163
Acknowledgments 164
References 164
9.A Data Quality Indicators in SDR2 166
10 Harmonization of Panel Surveys: The Cross-National Equivalent File 169
Dean R. Lillard
10.1 Introduction 169
10.2 Applied Harmonization Methods 170
10.2.1 CNEF Country Data Sources, Current and Planned 176
10.3 Current CNEF Partners 176
10.3.1 The HILDA Survey 176
10.3.2 The SLID 176
10.3.3 The CFPS 177
10.3.4 The SOEP 177
10.3.4.1 The BHPS 177
10.3.4.2 Understanding Society, UKHLS 178
10.3.5 The ITA.LI 178
10.3.6 The JHPS 178
10.3.7 The RLMS-HSE 178
10.3.8 The KLIPS 179
10.3.9 The Swedish Pseudo-Panel 179
10.3.10 The SHP 179
10.3.11 The PSID 179
10.4 Planned CNEF Partners 180
10.4.1 The ASEP 180
10.4.2 LISA 180
10.4.3 The ILS 180
10.4.4 The MxFLS 180
10.4.5 The NIDS 181
10.4.6 The PSFD 181
10.5 Documentation and Quality Assurance 181
10.6 Challenges to Harmonization 183
10.7 Recommendations for Researchers Interested in Harmonizing Panel Survey Data 185
10.8 Conclusion 186
References 187
11 Harmonization of Survey Data from UK Longitudinal Studies: CLOSER 189
Dara O’Neill and Rebecca Hardy
11.1 Introduction 189
11.2 Applied Harmonization Methods 191
11.2.1 Occupational Social Class 191
11.2.2 Body Size/Anthropometric Data 193
11.2.3 Mental Health 194
11.2.4 Harmonization Methods: Divergence and Convergence 195
11.3 Documentation and Quality Assurance 196
11.4 Challenges to Harmonization 198
11.5 Software Tools 199
11.6 Recommendations 200
Acknowledgments 202
References 202
12 Harmonization of Census Data: IPUMS – International 207
Steven Ruggles, Lara Cleveland, and Matthew Sobek
12.1 Introduction 207
12.2 Project History 208
12.2.1 Evolution of the Web Dissemination System 210
12.3 Applied Harmonization Methods 210
12.4 Documentation and Quality Assurance 215
12.5 Challenges to Harmonization 217
12.6 Software Tools 221
12.6.1 Metadata Tools 221
12.6.2 Data Reformatting 221
12.6.3 Data Harmonization 221
12.6.4 Dissemination System 222
12.7 Team Organization and Project Management 222
12.8 Lessons and Recommendations 223
References 225
Part III Domain-driven ex-post harmonization 227
13 Maelstrom Research Approaches to Retrospective Harmonization of Cohort Data for Epidemiological Research 229
Tina W. Wey and Isabel Fortier
13.1 Introduction 229
13.2 Applied Harmonization Methods 230
13.2.1 Implementing the Project 233
13.2.1.1 Initiating Activities and Organizing the Operational Framework 233
13.2.1.2 Assembling Study Information and Selecting Final Participating Studies (Guidelines Step 1) 234
13.2.1.3 Defining Target Variables to be Harmonized (the DataSchema) and Evaluating Harmonization Potential across Studies (Guidelines Step 2) 235
13.2.2 Producing the Harmonized Datasets 236
13.2.2.1 Processing Data (Guidelines Step 3a) 236
13.2.2.2 Processing Study-Specific Data to Generate Harmonized Datasets (Guidelines Step 3b) 237
13.3 Documentation and Quality Assurance 238
13.4 Challenges to Harmonization 240
13.5 Software Tools 241
13.6 Recommendations 243
Acknowledgments 244
References 245
14 Harmonizing and Synthesizing Partnership Histories from Different German Survey Infrastructures 249
Bernd Weiß, Sonja Schulz, Lisa Schmid, Sebastian Sterl, and Anna-Carolina Haensch
14.1 Introduction 249
14.2 Applied Harmonization Methods 250
14.2.1 Data Search Strategy and Data Access 250
14.2.2 Processing and Harmonizing Data 253
14.2.2.1 Harmonizing Partnership Biography Data 253
14.2.2.2 Harmonizing Additional Variables on Respondents’ or Couples’ Characteristics 254
14.3 Documentation and Quality Assurance 255
14.3.1 Documentation 255
14.3.2 Quality Assurance 256
14.3.2.1 Process-Related Quality Assurance 256
14.3.2.2 Benchmarking the Harmonized HaSpaD Data Set with Official Statistics 256
14.4 Challenges to Harmonization 258
14.4.1 Analyzing Harmonized Complex Survey Data 258
14.4.2 Sporadically and Systematically Missing Data 259
14.5 Software Tools 260
14.6 Recommendations 262
14.6.1 Harmonizing Biographical Data 262
14.6.1.1 Methodological Recommendations 262
14.6.1.2 Procedural Recommendations 263
14.6.1.3 Technical Recommendations 263
14.6.2 Getting Started with the Cumulative HaSpaD Data Set 263
Acknowledgments 264
References 264
15 Harmonization and Quality Assurance of Income and Wealth Data: The Case of LIS 269
Jörg Neugschwender, Teresa Munzi, and Piotr R. Paradowski
15.1 Introduction 269
15.2 Applied Harmonization Methods 271
15.3 Documentation and Quality Assurance 275
15.3.1 Quality Assurance 275
Selection of Source Datasets 276
Harmonization 276
Validation – “Green Light” Check 276
15.3.2 Documentation 278
15.4 Challenges to Harmonization 278
15.5 Software Tools 281
15.6 Conclusion 282
References 283
16 Ex-Post Harmonization of Time Use Data: Current Practices and Challenges in the Field 285
Ewa Jarosz, Sarah Flood, and Margarita Vega-Rapun
16.1 Introduction 285
16.2 Applied Harmonization Methods 289
16.2.1 Harmonizing the Matrix of the Diary 289
16.2.2 Variable Harmonization 291
16.2.3 Other Variables 293
16.2.4 Other Types of Time Use Data 294
16.3 Documentation and Quality Assurance 294
16.3.1 Documentation 294
16.3.2 Quality Checks 296
16.4 Challenges to Harmonization 297
16.5 Software Tools 300
16.6 Recommendations 301
References 302
Part IV Further Issues: Dealing with Methodological Issues in Harmonized Survey Data 305
17 Assessing and Improving the Comparability of Latent Construct Measurements in Ex-Post Harmonization 307
Ranjit K. Singh and Markus Quandt
17.1 Introduction 307
17.2 Measurement and Reality 307
17.3 Construct Match 308
17.3.1 Consequences of a Mismatch 309
17.3.2 Assessment 309
17.3.2.1 Qualitative Research Methods 309
17.3.2.2 Construct and Criterion Validity 309
17.3.2.3 Techniques for Multi-Item Instruments 310
17.3.2.4 Improving Construct Comparability 311
17.4 Reliability Differences 311
17.4.1 Consequences of Reliability Differences 311
17.4.2 Assessment 312
17.4.3 Improving Reliability Comparability 312
17.5 Units of Measurement 312
17.5.1 Consequences of Unit Differences 313
17.5.2 Improving Unit Comparability 313
17.5.3 Controlling for Instrument Characteristics 314
17.5.4 Harmonizing Units Based on Repeated Measurements 315
17.5.5 Harmonizing Units Based on Measurements Obtained from the Same Population 315
17.6 Cross-Cultural Comparability 316
17.6.1 Construct Match 316
17.6.1.1 Translation and Cognitive Probing 317
17.6.2 Reliability 317
17.6.3 Units of Measurement 318
17.6.3.1 Harmonizing Units of Localized Versions of the Same Instrument 318
17.6.3.2 Harmonizing Units Across Cultures and Instruments 318
17.6.4 Cross-Cultural Comparability of Multi-Item Instruments 318
17.7 Discussion and Outlook 319
References 320
18 Comparability and Measurement Invariance 323
Artur Pokropek
18.1 Latent Variable Framework for Testing and Accounting for Measurement Non-Invariance 324
18.2 Approaches to Empirical Assessment of Measurement Equivalence 325
18.2.1 Classical Invariance Analysis (MG-CFA) 326
18.2.2 Partial Invariance (MG-CFA) 327
18.2.3 Approximate Invariance 327
18.2.4 Approximate Partial Invariance (Alignment, BSEM Alignment, Partial BSEM) 328
18.3 Beyond Multiple Indicators 329
18.4 Conclusions 329
References 330
19 On the Creation, Documentation, and Sensible Use of Weights in the Context of Comparative Surveys 333
Dominique Joye, Marlène Sapin, and Christof Wolf
19.1 Introduction 333
19.2 Design Weights 335
19.2.1 What to do? 336
19.3 Post-stratification Weights 337
19.3.1 What Should be Done? 340
19.4 Population Weights 341
19.4.1 What Should be Done? 342
19.5 Conclusion 342
References 344
20 On Using Harmonized Data in Statistical Analysis: Notes of Caution 347
Claire Durand
20.1 Introduction 347
20.2 Challenges in the Combination of Data Sets 347
20.2.1 A First Principle: A No Censorship Inclusive Approach 348
20.2.2 A Second Principle: Using Multilevel Analysis and Introducing a Measurement Level 349
20.2.3 A Third Principle: Assessing the Equivalence of Survey Projects 351
20.3 Challenges in the Analysis of Combined Data Sets 353
20.3.1 Dealing with Time 354
20.3.2 Dealing with Missing Values 358
20.3.2.1 Missing Values at the Respondent and Measurement Level 358
20.3.2.2 Missing Values at the Survey Level 359
20.3.3 Dealing with Weights 361
20.4 Recommendations 362
References 363
21 On the Future of Survey Data Harmonization 367
Kazimierz M. Slomczynski, Christof Wolf, Irina Tomescu-Dubrow, and J. Craig Jenkins
21.1 What We Have Learned from Contributions on Survey Data Harmonization in this Volume 368
21.2 New Opportunities and Challenges 370
21.2.1 Reorientation of Survey Research in the Era of New Technology 370
21.2.2 Advances in Technical Aspects of Data Management 370
21.2.3 Harmonizing Survey Data with Other Types of Data 371
21.3 Developing a New Methodology of Harmonizing Non-Survey Data 372
21.3.1 Emerging Legal and Ethical Issues 372
21.4 Globalization of Science and Harmonizing Scientific Practice 373
References 373
Index 377
About the Author :
Irina Tomescu-Dubrow is Professor of Sociology at the Institute of Philosophy and Sociology at the Polish Academy of Sciences (PAN), and director of the Graduate School for Social Research at PAN.
Christof Wolf is President of GESIS — Leibniz Institute for the Social Sciences and Professor of Sociology at the University of Mannheim in Germany.
Kazimierz M. Slomczynski is Professor of Sociology at the Institute of Philosophy and Sociology, the Polish Academy of Sciences (IFiS PAN) and Academy Professor of Sociology at the Ohio State University (OSU). He co-directs CONSIRT - the Cross-national Studies: Interdisciplinary Research and Training program at OSU and IFiS PAN.
J. Craig Jenkins is Academy Professor of Sociology and Senior Research Scientist at the Mershon Center for International Security at the Ohio State University.
