Structural Equation Modeling of Multiple Rater Data

The use of multiple raters can improve the validity of conclusions made on self- (and other) reports of emotions, attitudes, goals, and self-perceptions of personality. Yet analyzing these ratings requires special psychometric models that take into account the specific nature of these data. From leading authorities, this book offers the first comprehensive introduction to structural equation modeling (SEM) of multiple rater data. Rather than taking a one-size-fits-all approach, the book shows how the choice of a model should be guided by measurement design and purpose. Practical recommendations are provided for selecting suitable measurement designs, raters, and psychometric models. Models for different combinations of rater types and for cross-sectional as well as longitudinal research designs are described step by step, with a strong emphasis on the substantive meaning of the latent variables in the models. User-friendly features include equation boxes, application boxes, and a companion website with Mplus and lavaan code for the book’s examples.

Table of Contents:
1. Introduction: The Importance of Multiple Rater Data 1.1 Advantages and Limitations of Self-Reports 1.2 Advantages and Limitations of Other Reports 1.21 Models of accuracy of other ratings 1.22 Sources of accuracy of other ratings 1.3 Usefulness of Multiple Rater Studies 1.31 Analyzing the validity of ratings 1.32 Improving the validity of inferences by multiple ratings 1.4 The Role of Measurement Models 1.5 Summary 1.6 Suggested Further Readings 2. Basic Methodological Concepts 2.1 Design Issues 2.1.1 Interchangeable and structurally different raters 2.1.2 Measurement designs 2.2 Confirmatory Factor Analysis of Multiple Rater Data 2.3 Stochastic Measurement Theory: Basic Ideas 2.3.1 Sampling process for structurally different raters 2.3.2 Sampling process for interchangeable raters 2.3.3 Differences between structurally different and interchangeable raters 2.4 Overview of the Present Book 2.5 Summary 2.6 Suggested Further Readings 3. Basic Models for Structurally Different Raters 3.1 Basic Decomposition of Observed Variables 3.2 Basic Model with Correlated First-Order Factors 3.2.1 Application of the MTMR model with correlated first-order factors: Loneliness and flourishing 3.3 Model with Indicator-Specific Factors 3.3.1 Application of the MTMR model with correlated first-order factors and indicator-specific factors: Loneliness and flourishing 3.3.2 Recommendations: Model Selection 3.4 Basic Model with Measurement Invariance Across Raters 3.4.1 Statistical tests for testing measurement invariance 3.4.2 Partial measurement invariance and measurement invariance in models with indicator-specific factors 3.4.3 How important is measurement invariance in multirater studies? 3.5 Application of the Models to Other Measurement Designs 3.6 Chapter Summary 3.7 Suggested Further Readings 4. Models with Method Factors for Structurally Different Raters 4.1 Basic CTC(M-1) Model 4.1.1 Choice of a reference rater group 4.1.2 Application of the CTC(M-1) model: Loneliness and flourishing 4.1.3 Comparing the CTC(M-1) model with the model with correlated first-order factors 4.2 Restricted CTC(M-1) Model as Reformulation of the Model with Correlated First-Order Factors 4.3 Restricted CTC(M-1) Model with Measurement Invariance Across Raters 4.4 CTC(M-1) Models with Indicator-Specific Effects 4.4.1 CTC(M-1) models with indicator-specific factors 4.4.2 CTC(M-1) model with indicator-specific traits 4.4.3. Choosing a CTC(M-1) model with indicator-specific effects 4.5 Alternative Models 4.5.1 Latent difference model 4.5.2 Latent means model 4.5.3 Reference rating as outcome model 4.6 Models with Covariates 4.7 Application of the Models to Other Measurement Designs 4.7.1 Analyzing round-robin data for structurally different raters 4.8 Chapter Summary 4.9 Suggested Further Readings 5. Single-Level CFA Models for Interchangeable Raters 5.1 The Interchangeable-Saturated (I-SAT) Model 5.2 Basic Decomposition 5.3 Basic Models with Correlated First-Order Factors 5.3.1 Adjustment of fit statistics with interchangeable models 5.3.2 Application of the basic model with correlated first-order factors with and without indicator-specific factors: Loneliness and flourishing 5.4 Models with Method Factors 5.4.1 Basic correlated traits-interchangeable methods (CTIM) model 5.4.2 Application of the basic CTIM model: Loneliness and flourishing 5.4.3 Restricted CTIM model 5.4.4 Application of the restricted CTIM model: Loneliness and flourishing 5.4.5 CTIM models with indicator-specific effects 5.5 Chapter Summary 5.6 Suggested Further Readings 6. Multilevel CFA Models for Interchangeable Raters 6.1 Multilevel Confirmatory Factor Analysis 6.1.1 Wide- versus long-format data 6.2 Basic Models with Correlated First-Order Factors and CTIM Models 6.2.1 Invariance constraints, variance decomposition, and coefficients 6.2.2 Application of CTIM models: Loneliness and flourishing 6.3 Practical Issues 6.4 Models for Round-Robin Designs with Interchangeable Raters 6.4.1 Analyzing round-robin data with interchangeable raters 6.4.2 Application to flourishing and loneliness data 6.5 Chapter Summary 6.6 Suggested Further Readings 7. Models for a Combination of Structurally Different and Interchangeable Raters 7.1 Basic Multilevel Models with Correlated First-Order Factors for a Traditional Combination of Different Types of Raters 7.2 Multilevel CTC(M-1) Models for a Combination of Structurally Different and Interchangeable Rate 7.2.1 Application of the multilevel CTC(M-1) model: Loneliness and flourishing 7.2.2 The multilevel CTC(M-1) model as an extension of the trait-reputation-identity model to multiple traits 7.3 Extensions of the models to other measurement designs 7.3.1 Multilevel CTC(M-1) model for multiple sets of interchangeable raters 7.3.2 Multilevel CTC(M-1) model for fully nested structurally different raters 7.3.3 Application of CFA-MTMR models for a special combination of rater groups 7.4 Chapter Summary 7.5 Suggested Further Readings 8.Models for Cross-Classified Multiple Membership Data 8.1 Models for Crossed Designs with Overlapping Sets of Interchangeable Raters 8.2 Models for Crossed Designs with a Combination of Structurally Different and Overlapping Sets of Interchangeable Raters 8.2.1 Application of the cross-classified CTC(M-1) model: Teaching quality and engagement 8.3 Extensions of the models to other measurement designs 8.4 Chapter Summary 8.5 Suggested Further Readings 9. Models for Longitudinal Multirater Data 9.1 Basic Decomposition of Observed Variables 9.2 Basic Multistate-Multirater (MSMR) Model 9.2.1 Testing for measurement invariance across time 9.2.2 Longitudinal Modeling Strategy 9.2.3 Application of the MSMR Model: Loneliness and flourishing 9.2.4 Summary and extensions 9.3 The Correlated States-Correlated (Methods – 1) [CSC(M – 1)] Model 9.3.1 Application of the CSM (M-1) Model with Indicator-Specific Factors: Loneliness and Flourishing 9.4 The CSC(M-1) Change Model 9.4.1 Application of the CSC(M -1) change model: Loneliness and flourishing 9.5 The Multimethod LST Model 9.5.1 Application of the MM-LST model: Loneliness and flourishing 9.6 Longitudinal Multirater Models for a Combination of Structurally Different and Interchangeable Raters 9.6.1 The latent state combination of methods (LS-COM) model 9.6.2 Application of the LS-COM model 9.6.3 The latent state-trait combination of methods (LST-COM) model 9.6.4 Application of the LST-COM model 9.7 Chapter Summary 9.8 Suggested Further Readings 10. Advanced Topics in Multitrait-Multirater Analysis 10.1 CFA-MTMR Models with Categorical Observed Variables 10.1.1 Practical recommendations 10.1.2 Application of CFA-MTMR models with categorical observed variables 10.2 Explaining Latent Trait and Rater-Specific Factors in CFA-MTMR Models without Bias 10.2.1 Mean structure 10.2.2 Covariance structure 10.3 Predicting External Criterion Variables based on CFA-MTMR Models 10.3.1 Models with only structurally different raters 10.3.2 Models with interchangeable raters 10.3.3 Application 10.4 Chapter Summary 10.5 Suggested Further Readings 11. Recommendations and Outlook 11.1 Choice of a Design 11.1.1 Type

About the Author :
Michael Eid, PhD, is Professor of Methods and Evaluation at the Free University of Berlin in Germany. His research focuses on measurement theory, in particular on the development of psychometric models for longitudinal and multimethod research. Since the early 2000s, he has been contributing to the development of structural equation models for analyzing multirater data for different types of raters and research designs. His more applied research contributions are in the areas of subjective well-being, mood regulation, and health psychology. Christian Geiser, PhD, is a former professor of quantitative psychology. He currently works as an instructor and statistical consultant with QuantFish. His areas of expertise are in structural equation modeling, measurement, longitudinal data analysis, latent class modeling, and multitrait–multimethod analysis. Tobias Koch, PhD, is Professor of Psychological Methods at the Friedrich-Schiller-Universität Jena in Germany. His research focuses on measurement theory and psychometrics, structural equation modeling, longitudinal data analysis, multilevel analysis, Bayesian analysis, and multitrait–multimethod analysis.

Review :
"This book provides a comprehensive yet accessible overview of the analysis of data from multiple raters. It introduces readers to the most basic models for analyzing multiple rater data, while also providing in-depth coverage of more advanced models that can be used to analyze more complex designs and answer more complex questions."--Richard E. Lucas, PhD, MSU Foundation Professor, Department of Psychology, Michigan State University "This is a great book for advanced graduate students who are well grounded in statistics and seek a better understanding of how models for multiple raters work. The book does a good job of introducing each of the models and how they are estimated."--Sara Tomek, PhD, Department of Educational Psychology, Baylor University "Written by leading scholars, this book fills a gap in the literature by providing a coherent, concise account of models for analyzing multiple rater data within the CFA framework. The authors present an up-to-date, self-contained introduction to the field. I would recommend this book to students and colleagues and consider it for teaching courses on CFA modeling."--Thomas Eckes, PhD, TestDaF Institute (retired), Bochum, Germany "The extensive coverage will make this the go-to reference on the analysis of multiple rater data."--David Kaplan, PhD, Hilldale Professor and Patricia Busk Professor of Quantitative Methods, Department of Educational Psychology, University of Wisconsin–Madison-