Multiblock Data Fusion in Statistics and Machine Learning: Applications in the Natural and Life Sciences

Multiblock Data Fusion in Statistics and Machine Learning Explore the advantages and shortcomings of various forms of multiblock analysis, and the relationships between them, with this expert guide Arising out of fusion problems that exist in a variety of fields in the natural and life sciences, the methods available to fuse multiple data sets have expanded dramatically in recent years. Older methods, rooted in psychometrics and chemometrics, also exist. Multiblock Data Fusion in Statistics and Machine Learning: Applications in the Natural and Life Sciences is a detailed overview of all relevant multiblock data analysis methods for fusing multiple data sets. It focuses on methods based on components and latent variables, including both well-known and lesser-known methods with potential applications in different types of problems. Many of the included methods are illustrated by practical examples and are accompanied by a freely available R-package. The distinguished authors have created an accessible and useful guide to help readers fuse data, develop new data fusion models, discover how the involved algorithms and models work, and understand the advantages and shortcomings of various approaches. This book includes: A thorough introduction to the different options available for the fusion of multiple data sets, including methods originating in psychometrics and chemometrics Practical discussions of well-known and lesser-known methods with applications in a wide variety of data problems Included, functional R-code for the application of many of the discussed methods Perfect for graduate students studying data analysis in the context of the natural and life sciences, including bioinformatics, sensometrics, and chemometrics, Multiblock Data Fusion in Statistics and Machine Learning: Applications in the Natural and Life Sciences is also an indispensable resource for developers and users of the results of multiblock methods.

Table of Contents:
Foreword xiii Preface xv List of Figures xvii List of Tables xxxi Part I Introductory Concepts and Theory 1 1 Introduction 3 1.1 Scope of the Book 3 1.2 Potential Audience 4 1.3 Types of Data and Analyses 5 1.3.1 Supervised and Unsupervised Analyses 5 1.3.2 High-, Mid- and Low-level Fusion 5 1.3.3 Dimension Reduction 7 1.3.4 Indirect Versus Direct Data 8 1.3.5 Heterogeneous Fusion 8 1.4 Examples 8 1.4.1 Metabolomics 8 1.4.2 Genomics 11 1.4.3 Systems Biology 13 1.4.4 Chemistry 13 1.4.5 Sensory Science 15 1.5 Goals of Analyses 16 1.6 Some History 17 1.7 Fundamental Choices 17 1.8 Common and Distinct Components 19 1.9 Overview and Links 20 1.10 Notation and Terminology 21 1.11 Abbreviations 22 2 Basic Theory and Concepts 25 2.i General Introduction 25 2.1 Component Models 25 2.1.1 General Idea of Component Models 25 2.1.2 Principal Component Analysis 26 2.1.3 Sparse PCA 30 2.1.4 Principal Component Regression 31 2.1.5 Partial Least Squares 32 2.1.6 Sparse PLS 36 2.1.7 Principal Covariates Regression 37 2.1.8 Redundancy Analysis 38 2.1.9 Comparing PLS, PCovR and RDA 38 2.1.10 Generalised Canonical Correlation Analysis 38 2.1.11 Simultaneous Component Analysis 39 2.2 Properties of Data 39 2.2.1 Data Theory 39 2.2.2 Scale-types 42 2.3 Estimation Methods 44 2.3.1 Least-squares Estimation 44 2.3.2 Maximum-likelihood Estimation 45 2.3.3 Eigenvalue Decomposition-based Methods 47 2.3.4 Covariance or Correlation-based Estimation Methods 47 2.3.5 Sequential Versus Simultaneous Methods 48 2.3.6 Homogeneous Versus Heterogeneous Fusion 50 2.4 Within- and Between-block Variation 52 2.4.1 Definition and Example 52 2.4.2 MAXBET Solution 54 2.4.3 MAXNEAR Solution 54 2.4.4 PLS2 Solution 55 2.4.5 CCA Solution 55 2.4.6 Comparing the Solutions 56 2.4.7 PLS, RDA and CCA Revisited 56 2.5 Framework for Common and Distinct Components 60 2.6 Preprocessing 63 2.7 Validation 64 2.7.1 Outliers 64 2.7.1.1 Residuals 64 2.7.1.2 Leverage 66 2.7.2 Model Fit 67 2.7.3 Bias-variance Trade-off 69 2.7.4 Test Set Validation 70 2.7.5 Cross-validation 72 2.7.6 Permutation Testing 75 2.7.7 Jackknife and Bootstrap 76 2.7.8 Hyper-parameters and Penalties 77 2.8 Appendix 78 3 Structure of Multiblock Data 87 3.i General Introduction 87 3.1 Taxonomy 87 3.2 Skeleton of a Multiblock Data Set 87 3.2.1 Shared Sample Mode 88 3.2.2 Shared Variable Mode 88 3.2.3 Shared Variable or Sample Mode 88 3.2.4 Shared Variable and Sample Mode 89 3.3 Topology of a Multiblock Data Set 90 3.3.1 Unsupervised Analysis 90 3.3.2 Supervised Analysis 93 3.4 Linking Structures 95 3.4.1 Linking Structure for Unsupervised Analysis 95 3.4.2 Linking Structures for Supervised Analysis 96 3.5 Summary 98 4 Matrix Correlations 99 4.i General Introduction 99 4.1 Definition 99 4.2 Most Used Matrix Correlations 101 4.2.1 Inner Product Correlation 101 4.2.2 GCD coefficient 101 4.2.3 RV-coefficient 102 4.2.4 SMI-coefficient 102 4.3 Generic Framework of Matrix Correlations 104 4.4 Generalised Matrix Correlations 105 4.4.1 Generalised RV-coefficient 105 4.4.2 Generalised Association Coefficient 106 4.5 Partial Matrix Correlations 108 4.6 Conclusions and Recommendations 110 4.7 Open Issues 111 Part II Selected Methods for Unsupervised and Supervised Topologies 113 5 Unsupervised Methods 115 5.i General Introduction 115 5.ii Relations to the General Framework 115 5.1 Shared Variable Mode 117 5.1.1 Only Common Variation 117 5.1.1.1 Simultaneous Component Analysis 117 5.1.1.2 Clustering and SCA 123 5.1.1.3 Multigroup Data Analysis 125 5.1.2 Common, Local, and Distinct Variation 126 5.1.2.1 Distinct and Common Components 127 5.1.2.2 Multivariate Curve Resolution 130 5.2 Shared Sample Mode 133 5.2.1 Only Common Variation 133 5.2.1.1 SUM-PCA 133 5.2.1.2 Multiple Factor Analysis and STATIS 135 5.2.1.3 Generalised Canonical Analysis 136 5.2.1.4 Regularised Generalised Canonical Correlation Analysis 139 5.2.1.5 Exponential Family SCA 140 5.2.1.6 Optimal-scaling 143 5.2.2 Common, Local, and Distinct Variation 146 5.2.2.1 Joint and Individual Variation Explained 146 5.2.2.2 Distinct and Common Components 147 5.2.2.3 PCA-GCA 148 5.2.2.4 Advanced Coupled Matrix and Tensor Factorisation 153 5.2.2.5 Penalised-ESCA 156 5.2.2.6 Multivariate Curve Resolution 158 5.3 Generic Framework 159 5.3.1 Framework for Simultaneous Unsupervised Methods 159 5.3.1.1 Description of the Framework 159 5.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods 161 5.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods 161 5.4 Conclusions and Recommendations 162 5.5 Open Issues 164 6 ASCA and Extensions 167 6.i General Introduction 167 6.ii Relations to the General Framework 167 6.1 ANOVA-Simultaneous Component Analysis 168 6.1.1 The ASCA Method 168 6.1.2 Validation of ASCA 176 6.1.2.1 Permutation Testing 176 6.1.2.2 Back-projection 178 6.1.2.3 Confidence Ellipsoids 178 6.1.3 The ASCA+ and LiMM-PCA Methods 181 6.2 Multilevel-SCA 182 6.3 Penalised-ASCA 183 6.4 Conclusions and Recommendations 185 6.5 Open Issues 186 7 Supervised Methods 187 7.i General Introduction 187 7.ii Relations to the General Framework 187 7.1 Multiblock Regression: General Perspectives 188 7.1.1 Model and Assumptions 188 7.1.2 Different Challenges and Aims 188 7.2 Multiblock PLS Regression 190 7.2.1 Standard Multiblock PLS Regression 190 7.2.2 MB-PLS Used for Classification 194 7.2.3 Sparse Multiblock PLS Regression (sMB-PLS) 196 7.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression) 199 7.3.1 The SO-PLS Method 199 7.3.2 Order of Blocks 202 7.3.3 Interpretation Tools 202 7.3.4 Restricted PLS Components and their Application in SO-PLS 203 7.3.5 Validation and Component Selection 204 7.3.6 Relations to ANOVA 205 7.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks 212 7.3.8 Further Applications of SO-PLS 215 7.3.9 Relations Between SO-PLS and ASCA 215 7.4 Parallel and Orthogonalised PLS (PO-PLS) Regression 217 7.5 Response Oriented Sequential Alternation 222 7.5.1 The ROSA Method 222 7.5.2 Validation 225 7.5.3 Interpretation 225 7.6 Conclusions and Recommendations 228 7.7 Open Issues 229 Part III Methods for Complex Multiblock Structures 231 8 Complex Block Structures; with Focus on L-Shape Relations 233 8.i General Introduction 233 8.ii Relations to the General Framework 234 8.1 Analysis of L-shape Data: General Perspectives 235 8.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA 236 8.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First 236 8.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First 238 8.2.3 Analysis of Segments/Clusters of X1 Data 240 8.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data 246 8.3.1 The Original L-PLS Method, Endo-L-PLS 247 8.3.2 Exo- Versus Endo-L-PLS 250 8.4 Modifications of the Original L-PLS Idea 252 8.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter α252 8.4.2 Three-blocks Bifocal PLS 253 8.5 Alternative L-shape Data Analysis Methods 254 8.5.1 Principal Component Analysis with External Information 254 8.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration 255 8.5.3 Multivariate Curve Resolution for Incomplete Data 256 8.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1 257 8.6 Domino PLS and More Complex Data Structures 258 8.7 Conclusions and Recommendations 258 8.8 Open Issues 260 Part IV Alternative Methods for Unsupervised and Supervised Topologies 261 9 Alternative Unsupervised Methods 263 9.i General Introduction 263 9.ii Relationship to the General Framework 263 9.1 Shared Variable Mode 263 9.2 Shared Sample Mode 265 9.2.1 Only Common Variation 265 9.2.1.1 DIABLO 265 9.2.1.2 Generalised Coupled Tensor Factorisation 266 9.2.1.3 Representation Matrices 267 9.2.1.4 Extended PCA 272 9.2.2 Common, Local, and Distinct Variation 273 9.2.2.1 Generalised SVD 273 9.2.2.2 Structural Learning and Integrative Decomposition 273 9.2.2.3 Bayesian Inter-battery Factor Analysis 275 9.2.2.4 Group Factor Analysis 276 9.2.2.5 OnPLS 277 9.2.2.6 Generalised Association Study 278 9.2.2.7 Multi-Omics Factor Analysis 278 9.3 Two Shared Modes and Only Common Variation 281 9.3.1 Generalised Procrustes Analysis 282 9.3.2 Three-way Methods 282 9.4 Conclusions and Recommendations 283 9.4.1 Open Issues 284 10 Alternative Supervised Methods 287 10.i General Introduction 287 10.ii Relations to the General Framework 287 10.1 Model and Focus 288 10.2 Extension of PCovR 288 10.2.1 Sparse Multiblock Principal Covariates Regression, Sparse PCovR 288 10.2.2 Multiway Multiblock Covariates Regression 289 10.3 Multiblock Redundancy Analysis 292 10.3.1 Standard Multiblock Redundancy Analysis 292 10.3.2 Sparse Multiblock Redundancy Analysis 294 10.4 Miscellaneous Multiblock Regression Methods 295 10.4.1 Multiblock Variance Partitioning 296 10.4.2 Network Induced Supervised Learning 296 10.4.3 Common Dimensions for Multiblock Regression 298 10.5 Modifications and Extensions of the SO-PLS Method 298 10.5.1 Extensions of SO-PLS to Three-Way Data 298 10.5.2 Variable Selection for SO-PLS 299 10.5.3 More Complicated Error Structure for SO-PLS 299 10.5.4 SO-PLS Used for Path Modelling 300 10.6 Methods for Data Sets Split Along the Sample Mode, Multigroup Methods 304 10.6.1 Multigroup PLS Regression 304 10.6.2 Clustering of Observations in Multiblock Regression 306 10.6.3 Domain-Invariant PLS, DI-PLS 307 10.7 Conclusions and Recommendations 308 10.8 Open Issues 309 Part V Software 311 11 Algorithms and Software 313 11.1 Multiblock Software 313 11.2 R package multiblock 313 11.3 Installing and Starting the Package 314 11.4 Data Handling 314 11.4.1 Read From File 314 11.4.2 Data Pre-processing 315 11.4.3 Re-coding Categorical Data 316 11.4.4 Data Structures for Multiblock Analysis 317 11.4.4.1 Create List of Blocks 317 11.4.4.2 Create data.frame of Blocks 317 11.5 Basic Methods 318 11.5.1 Prepare Data 319 11.5.2 Modelling 319 11.5.3 Common Output Elements Across Methods 319 11.5.4 Scores and Loadings 320 11.6 Unsupervised Methods 321 11.6.1 Formatting Data for Unsupervised Data Analysis 321 11.6.2 Method Interfaces 322 11.6.3 Shared Sample Mode Analyses 322 11.6.4 Shared Variable Mode 322 11.6.5 Common Output Elements Across Methods 323 11.6.6 Scores and Loadings 324 11.6.7 Plot From Imported Package 325 11.7 ANOVA Simultaneous Component Analysis 325 11.7.1 Formula Interface 325 11.7.2 Simulated Data 325 11.7.3 ASCA Modelling 325 11.7.4 ASCA Scores 326 11.7.5 ASCA Loadings 326 11.8 Supervised Methods 327 11.8.1 Formatting Data for Supervised Analyses 327 11.8.2 Multiblock Partial Least Squares 328 11.8.2.1 MB-PLS Modelling 328 11.8.2.2 MB-PLS Summaries and Plotting 328 11.8.3 Sparse Multiblock Partial Least Squares 328 11.8.3.1 Sparse MB-PLS Modelling 328 11.8.3.2 Sparse MB-PLS Plotting 329 11.8.4 Sequential and Orthogonalised Partial Least Squares 330 11.8.4.1 SO-PLS Modelling 330 11.8.4.2 Måge Plot 331 11.8.4.3 SO-PLS Loadings 332 11.8.4.4 SO-PLS Scores 333 11.8.4.5 SO-PLS Prediction 334 11.8.4.6 SO-PLS Validation 334 11.8.4.7 Principal Components of Predictions 336 11.8.4.8 CVANOVA 336 11.8.5 Parallel and Orthogonalised Partial Least Squares 337 11.8.5.1 PO-PLS Modelling 337 11.8.5.2 PO-PLS Scores and Loadings 338 11.8.6 Response Optimal Sequential Alternation 339 11.8.6.1 ROSA Modelling 339 11.8.6.2 ROSA Loadings 340 11.8.6.3 ROSA Scores 340 11.8.6.4 ROSA Prediction 340 11.8.6.5 ROSA Validation 341 11.8.6.6 ROSA Image Plots 342 11.8.7 Multiblock Redundancy Analysis 343 11.8.7.1 MB-RDA Modelling 343 11.8.7.2 MB-RDA Loadings and Scores 343 11.9 Complex Data Structures 344 11.9.1 L-PLS 344 11.9.1.1 Simulated L-shaped Data 344 11.9.1.2 Exo-L-PLS 344 11.9.1.3 Endo-L-PLS 344 11.9.1.4 L-PLS Cross-validation 345 11.9.2 SO-PLS-PM 345 11.9.2.1 Single SO-PLS-PM Model 346 11.9.2.2 Multiple Paths in an SO-PLS-PM Model 346 11.10 Software Packages 347 11.10.1 R Packages 347 11.10.2 MATLAB Toolboxes 348 11.10.3 Python 349 11.10.4 Commercial Software 349                References 351 Index 373