Age K. Smilde is a Professor of Biosystems Data Analysis at the Swammerdam Institute for Life Sciences at the University of Amsterdam. He also holds a part-time position at the Department of Machine Intelligence of Simula Metropolitan Center for Digital Engineering in Oslo, Norway. His research interest is multiblock data analysis and its implementation in different fields of life sciences. He is currently the Editor-in-Chief of the Journal of Chemometrics. Tormod Næs is a Senior Scientist at Nofima, a food research institute in Norway. He is also currently employed as adjoint professor at the Department of Food Science, University of Copenhagen, Denmark and as extraordinary professor at University of Stellenbosch, South Africa. His main research interest is multivariate analysis with special emphasis on applications in sensory science and spectroscopy. Kristian Hovde Liland is an Associate Professor with a top scientist scholarship in Data Science at the Norwegian University of Life Sciences and works in the areas of chemometrics, data analysis and machine learning. His main research is in linear prediction modelling, spectroscopy, and the transition between chemometrics and machine learning.

• Foreword xiiiPreface xvList of Figures xviiList of Tables xxxiPart I Introductory Concepts and Theory 11 Introduction 31.1 Scope of the Book 31.2 Potential Audience 41.3 Types of Data and Analyses 51.3.1 Supervised and Unsupervised Analyses 51.3.2 High-, Mid- and Low-level Fusion 51.3.3 Dimension Reduction 71.3.4 Indirect Versus Direct Data 81.3.5 Heterogeneous Fusion 81.4 Examples 81.4.1 Metabolomics 81.4.2 Genomics 111.4.3 Systems Biology 131.4.4 Chemistry 131.4.5 Sensory Science 151.5 Goals of Analyses 161.6 Some History 171.7 Fundamental Choices 171.8 Common and Distinct Components 191.9 Overview and Links 201.10 Notation and Terminology 211.11 Abbreviations 222 Basic Theory and Concepts 252.i General Introduction 252.1 Component Models 252.1.1 General Idea of Component Models 252.1.2 Principal Component Analysis 262.1.3 Sparse PCA 302.1.4 Principal Component Regression 312.1.5 Partial Least Squares 322.1.6 Sparse PLS 362.1.7 Principal Covariates Regression 372.1.8 Redundancy Analysis 382.1.9 Comparing PLS, PCovR and RDA 382.1.10 Generalised Canonical Correlation Analysis 382.1.11 Simultaneous Component Analysis 392.2 Properties of Data 392.2.1 Data Theory 392.2.2 Scale-types 422.3 Estimation Methods 442.3.1 Least-squares Estimation 442.3.2 Maximum-likelihood Estimation 452.3.3 Eigenvalue Decomposition-based Methods 472.3.4 Covariance or Correlation-based Estimation Methods 472.3.5 Sequential Versus Simultaneous Methods 482.3.6 Homogeneous Versus Heterogeneous Fusion 502.4 Within- and Between-block Variation 522.4.1 Definition and Example 522.4.2 MAXBET Solution 542.4.3 MAXNEAR Solution 542.4.4 PLS2 Solution 552.4.5 CCA Solution 552.4.6 Comparing the Solutions 562.4.7 PLS, RDA and CCA Revisited 562.5 Framework for Common and Distinct Components 602.6 Preprocessing 632.7 Validation 642.7.1 Outliers 642.7.1.1 Residuals 642.7.1.2 Leverage 662.7.2 Model Fit 672.7.3 Bias-variance Trade-off 692.7.4 Test Set Validation 702.7.5 Cross-validation 722.7.6 Permutation Testing 752.7.7 Jackknife and Bootstrap 762.7.8 Hyper-parameters and Penalties 772.8 Appendix 783 Structure of Multiblock Data 873.i General Introduction 873.1 Taxonomy 873.2 Skeleton of a Multiblock Data Set 873.2.1 Shared Sample Mode 883.2.2 Shared Variable Mode 883.2.3 Shared Variable or Sample Mode 883.2.4 Shared Variable and Sample Mode 893.3 Topology of a Multiblock Data Set 903.3.1 Unsupervised Analysis 903.3.2 Supervised Analysis 933.4 Linking Structures 953.4.1 Linking Structure for Unsupervised Analysis 953.4.2 Linking Structures for Supervised Analysis 963.5 Summary 984 Matrix Correlations 994.i General Introduction 994.1 Definition 994.2 Most Used Matrix Correlations 1014.2.1 Inner Product Correlation 1014.2.2 GCD coefficient 1014.2.3 RV-coefficient 1024.2.4 SMI-coefficient 1024.3 Generic Framework of Matrix Correlations 1044.4 Generalised Matrix Correlations 1054.4.1 Generalised RV-coefficient 1054.4.2 Generalised Association Coefficient 1064.5 Partial Matrix Correlations 1084.6 Conclusions and Recommendations 1104.7 Open Issues 111Part II Selected Methods for Unsupervised and Supervised Topologies 1135 Unsupervised Methods 1155.i General Introduction 1155.ii Relations to the General Framework 1155.1 Shared Variable Mode 1175.1.1 Only Common Variation 1175.1.1.1 Simultaneous Component Analysis 1175.1.1.2 Clustering and SCA 1235.1.1.3 Multigroup Data Analysis 1255.1.2 Common, Local, and Distinct Variation 1265.1.2.1 Distinct and Common Components 1275.1.2.2 Multivariate Curve Resolution 1305.2 Shared Sample Mode 1335.2.1 Only Common Variation 1335.2.1.1 SUM-PCA 1335.2.1.2 Multiple Factor Analysis and STATIS 1355.2.1.3 Generalised Canonical Analysis 1365.2.1.4 Regularised Generalised Canonical Correlation Analysis 1395.2.1.5 Exponential Family SCA 1405.2.1.6 Optimal-scaling 1435.2.2 Common, Local, and Distinct Variation 1465.2.2.1 Joint and Individual Variation Explained 1465.2.2.2 Distinct and Common Components 1475.2.2.3 PCA-GCA 1485.2.2.4 Advanced Coupled Matrix and Tensor Factorisation 1535.2.2.5 Penalised-ESCA 1565.2.2.6 Multivariate Curve Resolution 1585.3 Generic Framework 1595.3.1 Framework for Simultaneous Unsupervised Methods 1595.3.1.1 Description of the Framework 1595.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods 1615.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods 1615.4 Conclusions and Recommendations 1625.5 Open Issues 1646 ASCA and Extensions 1676.i General Introduction 1676.ii Relations to the General Framework 1676.1 ANOVA-Simultaneous Component Analysis 1686.1.1 The ASCA Method 1686.1.2 Validation of ASCA 1766.1.2.1 Permutation Testing 1766.1.2.2 Back-projection 1786.1.2.3 Confidence Ellipsoids 1786.1.3 The ASCA+ and LiMM-PCA Methods 1816.2 Multilevel-SCA 1826.3 Penalised-ASCA 1836.4 Conclusions and Recommendations 1856.5 Open Issues 1867 Supervised Methods 1877.i General Introduction 1877.ii Relations to the General Framework 1877.1 Multiblock Regression: General Perspectives 1887.1.1 Model and Assumptions 1887.1.2 Different Challenges and Aims 1887.2 Multiblock PLS Regression 1907.2.1 Standard Multiblock PLS Regression 1907.2.2 MB-PLS Used for Classification 1947.2.3 Sparse Multiblock PLS Regression (sMB-PLS) 1967.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression) 1997.3.1 The SO-PLS Method 1997.3.2 Order of Blocks 2027.3.3 Interpretation Tools 2027.3.4 Restricted PLS Components and their Application in SO-PLS 2037.3.5 Validation and Component Selection 2047.3.6 Relations to ANOVA 2057.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks 2127.3.8 Further Applications of SO-PLS 2157.3.9 Relations Between SO-PLS and ASCA 2157.4 Parallel and Orthogonalised PLS (PO-PLS) Regression 2177.5 Response Oriented Sequential Alternation 2227.5.1 The ROSA Method 2227.5.2 Validation 2257.5.3 Interpretation 2257.6 Conclusions and Recommendations 2287.7 Open Issues 229Part III Methods for Complex Multiblock Structures 2318 Complex Block Structures; with Focus on L-Shape Relations 2338.i General Introduction 2338.ii Relations to the General Framework 2348.1 Analysis of L-shape Data: General Perspectives 2358.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA 2368.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First 2368.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First 2388.2.3 Analysis of Segments/Clusters of X1 Data 2408.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data 2468.3.1 The Original L-PLS Method, Endo-L-PLS 2478.3.2 Exo- Versus Endo-L-PLS 2508.4 Modifications of the Original L-PLS Idea 2528.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter α2528.4.2 Three-blocks Bifocal PLS 2538.5 Alternative L-shape Data Analysis Methods 2548.5.1 Principal Component Analysis with External Information 2548.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration 2558.5.3 Multivariate Curve Resolution for Incomplete Data 2568.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1 2578.6 Domino PLS and More Complex Data Structures 2588.7 Conclusions and Recommendations 2588.8 Open Issues 260Part IV Alternative Methods for Unsupervised and Supervised Topologies 2619 Alternative Unsupervised Methods 2639.i General Introduction 2639.ii Relationship to the General Framework 2639.1 Shared Variable Mode 2639.2 Shared Sample Mode 2659.2.1 Only Common Variation 2659.2.1.1 DIABLO 2659.2.1.2 Generalised Coupled Tensor Factorisation 2669.2.1.3 Representation Matrices 2679.2.1.4 Extended PCA 2729.2.2 Common, Local, and Distinct Variation 2739.2.2.1 Generalised SVD 2739.2.2.2 Structural Learning and Integrative Decomposition 2739.2.2.3 Bayesian Inter-battery Factor Analysis 2759.2.2.4 Group Factor Analysis 2769.2.2.5 OnPLS 2779.2.2.6 Generalised Association Study 2789.2.2.7 Multi-Omics Factor Analysis 2789.3 Two Shared Modes and Only Common Variation 2819.3.1 Generalised Procrustes Analysis 2829.3.2 Three-way Methods 2829.4 Conclusions and Recommendations 2839.4.1 Open Issues 28410 Alternative Supervised Methods 28710.i General Introduction 28710.ii Relations to the General Framework 28710.1 Model and Focus 28810.2 Extension of PCovR 28810.2.1 Sparse Multiblock Principal Covariates Regression, Sparse PCovR 28810.2.2 Multiway Multiblock Covariates Regression 28910.3 Multiblock Redundancy Analysis 29210.3.1 Standard Multiblock Redundancy Analysis 29210.3.2 Sparse Multiblock Redundancy Analysis 29410.4 Miscellaneous Multiblock Regression Methods 29510.4.1 Multiblock Variance Partitioning 29610.4.2 Network Induced Supervised Learning 29610.4.3 Common Dimensions for Multiblock Regression 29810.5 Modifications and Extensions of the SO-PLS Method 29810.5.1 Extensions of SO-PLS to Three-Way Data 29810.5.2 Variable Selection for SO-PLS 29910.5.3 More Complicated Error Structure for SO-PLS 29910.5.4 SO-PLS Used for Path Modelling 30010.6 Methods for Data Sets Split Along the Sample Mode, Multigroup Methods 30410.6.1 Multigroup PLS Regression 30410.6.2 Clustering of Observations in Multiblock Regression 30610.6.3 Domain-Invariant PLS, DI-PLS 30710.7 Conclusions and Recommendations 30810.8 Open Issues 309Part V Software 31111 Algorithms and Software 31311.1 Multiblock Software 31311.2 R package multiblock 31311.3 Installing and Starting the Package 31411.4 Data Handling 31411.4.1 Read From File 31411.4.2 Data Pre-processing 31511.4.3 Re-coding Categorical Data 31611.4.4 Data Structures for Multiblock Analysis 31711.4.4.1 Create List of Blocks 31711.4.4.2 Create data.frame of Blocks 31711.5 Basic Methods 31811.5.1 Prepare Data 31911.5.2 Modelling 31911.5.3 Common Output Elements Across Methods 31911.5.4 Scores and Loadings 32011.6 Unsupervised Methods 32111.6.1 Formatting Data for Unsupervised Data Analysis 32111.6.2 Method Interfaces 32211.6.3 Shared Sample Mode Analyses 32211.6.4 Shared Variable Mode 32211.6.5 Common Output Elements Across Methods 32311.6.6 Scores and Loadings 32411.6.7 Plot From Imported Package 32511.7 ANOVA Simultaneous Component Analysis 32511.7.1 Formula Interface 32511.7.2 Simulated Data 32511.7.3 ASCA Modelling 32511.7.4 ASCA Scores 32611.7.5 ASCA Loadings 32611.8 Supervised Methods 32711.8.1 Formatting Data for Supervised Analyses 32711.8.2 Multiblock Partial Least Squares 32811.8.2.1 MB-PLS Modelling 32811.8.2.2 MB-PLS Summaries and Plotting 32811.8.3 Sparse Multiblock Partial Least Squares 32811.8.3.1 Sparse MB-PLS Modelling 32811.8.3.2 Sparse MB-PLS Plotting 32911.8.4 Sequential and Orthogonalised Partial Least Squares 33011.8.4.1 SO-PLS Modelling 33011.8.4.2 Måge Plot 33111.8.4.3 SO-PLS Loadings 33211.8.4.4 SO-PLS Scores 33311.8.4.5 SO-PLS Prediction 33411.8.4.6 SO-PLS Validation 33411.8.4.7 Principal Components of Predictions 33611.8.4.8 CVANOVA 33611.8.5 Parallel and Orthogonalised Partial Least Squares 33711.8.5.1 PO-PLS Modelling 33711.8.5.2 PO-PLS Scores and Loadings 33811.8.6 Response Optimal Sequential Alternation 33911.8.6.1 ROSA Modelling 33911.8.6.2 ROSA Loadings 34011.8.6.3 ROSA Scores 34011.8.6.4 ROSA Prediction 34011.8.6.5 ROSA Validation 34111.8.6.6 ROSA Image Plots 34211.8.7 Multiblock Redundancy Analysis 34311.8.7.1 MB-RDA Modelling 34311.8.7.2 MB-RDA Loadings and Scores 34311.9 Complex Data Structures 34411.9.1 L-PLS 34411.9.1.1 Simulated L-shaped Data 34411.9.1.2 Exo-L-PLS 34411.9.1.3 Endo-L-PLS 34411.9.1.4 L-PLS Cross-validation 34511.9.2 SO-PLS-PM 34511.9.2.1 Single SO-PLS-PM Model 34611.9.2.2 Multiple Paths in an SO-PLS-PM Model 34611.10 Software Packages 34711.10.1 R Packages 34711.10.2 MATLAB Toolboxes 34811.10.3 Python 34911.10.4 Commercial Software 349                References 351Index 373

Age K. Smilde is a Professor of Biosystems Data Analysis at the Swammerdam Institute for Life Sciences at the University of Amsterdam. He also holds a part-time position at the Department of Machine Intelligence of Simula Metropolitan Center for Digital Engineering in Oslo, Norway. His research interest is multiblock data analysis and its implementation in different fields of life sciences. He is currently the Editor-in-Chief of the Journal of Chemometrics. Tormod Næs is a Senior Scientist at Nofima, a food research institute in Norway. He is also currently employed as adjoint professor at the Department of Food Science, University of Copenhagen, Denmark and as extraordinary professor at University of Stellenbosch, South Africa. His main research interest is multivariate analysis with special emphasis on applications in sensory science and spectroscopy. Kristian Hovde Liland is an Associate Professor with a top scientist scholarship in Data Science at the Norwegian University of Life Sciences and works in the areas of chemometrics, data analysis and machine learning. His main research is in linear prediction modelling, spectroscopy, and the transition between chemometrics and machine learning.

• Foreword xiiiPreface xvList of Figures xviiList of Tables xxxiPart I Introductory Concepts and Theory 11 Introduction 31.1 Scope of the Book 31.2 Potential Audience 41.3 Types of Data and Analyses 51.3.1 Supervised and Unsupervised Analyses 51.3.2 High-, Mid- and Low-level Fusion 51.3.3 Dimension Reduction 71.3.4 Indirect Versus Direct Data 81.3.5 Heterogeneous Fusion 81.4 Examples 81.4.1 Metabolomics 81.4.2 Genomics 111.4.3 Systems Biology 131.4.4 Chemistry 131.4.5 Sensory Science 151.5 Goals of Analyses 161.6 Some History 171.7 Fundamental Choices 171.8 Common and Distinct Components 191.9 Overview and Links 201.10 Notation and Terminology 211.11 Abbreviations 222 Basic Theory and Concepts 252.i General Introduction 252.1 Component Models 252.1.1 General Idea of Component Models 252.1.2 Principal Component Analysis 262.1.3 Sparse PCA 302.1.4 Principal Component Regression 312.1.5 Partial Least Squares 322.1.6 Sparse PLS 362.1.7 Principal Covariates Regression 372.1.8 Redundancy Analysis 382.1.9 Comparing PLS, PCovR and RDA 382.1.10 Generalised Canonical Correlation Analysis 382.1.11 Simultaneous Component Analysis 392.2 Properties of Data 392.2.1 Data Theory 392.2.2 Scale-types 422.3 Estimation Methods 442.3.1 Least-squares Estimation 442.3.2 Maximum-likelihood Estimation 452.3.3 Eigenvalue Decomposition-based Methods 472.3.4 Covariance or Correlation-based Estimation Methods 472.3.5 Sequential Versus Simultaneous Methods 482.3.6 Homogeneous Versus Heterogeneous Fusion 502.4 Within- and Between-block Variation 522.4.1 Definition and Example 522.4.2 MAXBET Solution 542.4.3 MAXNEAR Solution 542.4.4 PLS2 Solution 552.4.5 CCA Solution 552.4.6 Comparing the Solutions 562.4.7 PLS, RDA and CCA Revisited 562.5 Framework for Common and Distinct Components 602.6 Preprocessing 632.7 Validation 642.7.1 Outliers 642.7.1.1 Residuals 642.7.1.2 Leverage 662.7.2 Model Fit 672.7.3 Bias-variance Trade-off 692.7.4 Test Set Validation 702.7.5 Cross-validation 722.7.6 Permutation Testing 752.7.7 Jackknife and Bootstrap 762.7.8 Hyper-parameters and Penalties 772.8 Appendix 783 Structure of Multiblock Data 873.i General Introduction 873.1 Taxonomy 873.2 Skeleton of a Multiblock Data Set 873.2.1 Shared Sample Mode 883.2.2 Shared Variable Mode 883.2.3 Shared Variable or Sample Mode 883.2.4 Shared Variable and Sample Mode 893.3 Topology of a Multiblock Data Set 903.3.1 Unsupervised Analysis 903.3.2 Supervised Analysis 933.4 Linking Structures 953.4.1 Linking Structure for Unsupervised Analysis 953.4.2 Linking Structures for Supervised Analysis 963.5 Summary 984 Matrix Correlations 994.i General Introduction 994.1 Definition 994.2 Most Used Matrix Correlations 1014.2.1 Inner Product Correlation 1014.2.2 GCD coefficient 1014.2.3 RV-coefficient 1024.2.4 SMI-coefficient 1024.3 Generic Framework of Matrix Correlations 1044.4 Generalised Matrix Correlations 1054.4.1 Generalised RV-coefficient 1054.4.2 Generalised Association Coefficient 1064.5 Partial Matrix Correlations 1084.6 Conclusions and Recommendations 1104.7 Open Issues 111Part II Selected Methods for Unsupervised and Supervised Topologies 1135 Unsupervised Methods 1155.i General Introduction 1155.ii Relations to the General Framework 1155.1 Shared Variable Mode 1175.1.1 Only Common Variation 1175.1.1.1 Simultaneous Component Analysis 1175.1.1.2 Clustering and SCA 1235.1.1.3 Multigroup Data Analysis 1255.1.2 Common, Local, and Distinct Variation 1265.1.2.1 Distinct and Common Components 1275.1.2.2 Multivariate Curve Resolution 1305.2 Shared Sample Mode 1335.2.1 Only Common Variation 1335.2.1.1 SUM-PCA 1335.2.1.2 Multiple Factor Analysis and STATIS 1355.2.1.3 Generalised Canonical Analysis 1365.2.1.4 Regularised Generalised Canonical Correlation Analysis 1395.2.1.5 Exponential Family SCA 1405.2.1.6 Optimal-scaling 1435.2.2 Common, Local, and Distinct Variation 1465.2.2.1 Joint and Individual Variation Explained 1465.2.2.2 Distinct and Common Components 1475.2.2.3 PCA-GCA 1485.2.2.4 Advanced Coupled Matrix and Tensor Factorisation 1535.2.2.5 Penalised-ESCA 1565.2.2.6 Multivariate Curve Resolution 1585.3 Generic Framework 1595.3.1 Framework for Simultaneous Unsupervised Methods 1595.3.1.1 Description of the Framework 1595.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods 1615.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods 1615.4 Conclusions and Recommendations 1625.5 Open Issues 1646 ASCA and Extensions 1676.i General Introduction 1676.ii Relations to the General Framework 1676.1 ANOVA-Simultaneous Component Analysis 1686.1.1 The ASCA Method 1686.1.2 Validation of ASCA 1766.1.2.1 Permutation Testing 1766.1.2.2 Back-projection 1786.1.2.3 Confidence Ellipsoids 1786.1.3 The ASCA+ and LiMM-PCA Methods 1816.2 Multilevel-SCA 1826.3 Penalised-ASCA 1836.4 Conclusions and Recommendations 1856.5 Open Issues 1867 Supervised Methods 1877.i General Introduction 1877.ii Relations to the General Framework 1877.1 Multiblock Regression: General Perspectives 1887.1.1 Model and Assumptions 1887.1.2 Different Challenges and Aims 1887.2 Multiblock PLS Regression 1907.2.1 Standard Multiblock PLS Regression 1907.2.2 MB-PLS Used for Classification 1947.2.3 Sparse Multiblock PLS Regression (sMB-PLS) 1967.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression) 1997.3.1 The SO-PLS Method 1997.3.2 Order of Blocks 2027.3.3 Interpretation Tools 2027.3.4 Restricted PLS Components and their Application in SO-PLS 2037.3.5 Validation and Component Selection 2047.3.6 Relations to ANOVA 2057.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks 2127.3.8 Further Applications of SO-PLS 2157.3.9 Relations Between SO-PLS and ASCA 2157.4 Parallel and Orthogonalised PLS (PO-PLS) Regression 2177.5 Response Oriented Sequential Alternation 2227.5.1 The ROSA Method 2227.5.2 Validation 2257.5.3 Interpretation 2257.6 Conclusions and Recommendations 2287.7 Open Issues 229Part III Methods for Complex Multiblock Structures 2318 Complex Block Structures; with Focus on L-Shape Relations 2338.i General Introduction 2338.ii Relations to the General Framework 2348.1 Analysis of L-shape Data: General Perspectives 2358.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA 2368.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First 2368.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First 2388.2.3 Analysis of Segments/Clusters of X1 Data 2408.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data 2468.3.1 The Original L-PLS Method, Endo-L-PLS 2478.3.2 Exo- Versus Endo-L-PLS 2508.4 Modifications of the Original L-PLS Idea 2528.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter α2528.4.2 Three-blocks Bifocal PLS 2538.5 Alternative L-shape Data Analysis Methods 2548.5.1 Principal Component Analysis with External Information 2548.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration 2558.5.3 Multivariate Curve Resolution for Incomplete Data 2568.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1 2578.6 Domino PLS and More Complex Data Structures 2588.7 Conclusions and Recommendations 2588.8 Open Issues 260Part IV Alternative Methods for Unsupervised and Supervised Topologies 2619 Alternative Unsupervised Methods 2639.i General Introduction 2639.ii Relationship to the General Framework 2639.1 Shared Variable Mode 2639.2 Shared Sample Mode 2659.2.1 Only Common Variation 2659.2.1.1 DIABLO 2659.2.1.2 Generalised Coupled Tensor Factorisation 2669.2.1.3 Representation Matrices 2679.2.1.4 Extended PCA 2729.2.2 Common, Local, and Distinct Variation 2739.2.2.1 Generalised SVD 2739.2.2.2 Structural Learning and Integrative Decomposition 2739.2.2.3 Bayesian Inter-battery Factor Analysis 2759.2.2.4 Group Factor Analysis 2769.2.2.5 OnPLS 2779.2.2.6 Generalised Association Study 2789.2.2.7 Multi-Omics Factor Analysis 2789.3 Two Shared Modes and Only Common Variation 2819.3.1 Generalised Procrustes Analysis 2829.3.2 Three-way Methods 2829.4 Conclusions and Recommendations 2839.4.1 Open Issues 28410 Alternative Supervised Methods 28710.i General Introduction 28710.ii Relations to the General Framework 28710.1 Model and Focus 28810.2 Extension of PCovR 28810.2.1 Sparse Multiblock Principal Covariates Regression, Sparse PCovR 28810.2.2 Multiway Multiblock Covariates Regression 28910.3 Multiblock Redundancy Analysis 29210.3.1 Standard Multiblock Redundancy Analysis 29210.3.2 Sparse Multiblock Redundancy Analysis 29410.4 Miscellaneous Multiblock Regression Methods 29510.4.1 Multiblock Variance Partitioning 29610.4.2 Network Induced Supervised Learning 29610.4.3 Common Dimensions for Multiblock Regression 29810.5 Modifications and Extensions of the SO-PLS Method 29810.5.1 Extensions of SO-PLS to Three-Way Data 29810.5.2 Variable Selection for SO-PLS 29910.5.3 More Complicated Error Structure for SO-PLS 29910.5.4 SO-PLS Used for Path Modelling 30010.6 Methods for Data Sets Split Along the Sample Mode, Multigroup Methods 30410.6.1 Multigroup PLS Regression 30410.6.2 Clustering of Observations in Multiblock Regression 30610.6.3 Domain-Invariant PLS, DI-PLS 30710.7 Conclusions and Recommendations 30810.8 Open Issues 309Part V Software 31111 Algorithms and Software 31311.1 Multiblock Software 31311.2 R package multiblock 31311.3 Installing and Starting the Package 31411.4 Data Handling 31411.4.1 Read From File 31411.4.2 Data Pre-processing 31511.4.3 Re-coding Categorical Data 31611.4.4 Data Structures for Multiblock Analysis 31711.4.4.1 Create List of Blocks 31711.4.4.2 Create data.frame of Blocks 31711.5 Basic Methods 31811.5.1 Prepare Data 31911.5.2 Modelling 31911.5.3 Common Output Elements Across Methods 31911.5.4 Scores and Loadings 32011.6 Unsupervised Methods 32111.6.1 Formatting Data for Unsupervised Data Analysis 32111.6.2 Method Interfaces 32211.6.3 Shared Sample Mode Analyses 32211.6.4 Shared Variable Mode 32211.6.5 Common Output Elements Across Methods 32311.6.6 Scores and Loadings 32411.6.7 Plot From Imported Package 32511.7 ANOVA Simultaneous Component Analysis 32511.7.1 Formula Interface 32511.7.2 Simulated Data 32511.7.3 ASCA Modelling 32511.7.4 ASCA Scores 32611.7.5 ASCA Loadings 32611.8 Supervised Methods 32711.8.1 Formatting Data for Supervised Analyses 32711.8.2 Multiblock Partial Least Squares 32811.8.2.1 MB-PLS Modelling 32811.8.2.2 MB-PLS Summaries and Plotting 32811.8.3 Sparse Multiblock Partial Least Squares 32811.8.3.1 Sparse MB-PLS Modelling 32811.8.3.2 Sparse MB-PLS Plotting 32911.8.4 Sequential and Orthogonalised Partial Least Squares 33011.8.4.1 SO-PLS Modelling 33011.8.4.2 Måge Plot 33111.8.4.3 SO-PLS Loadings 33211.8.4.4 SO-PLS Scores 33311.8.4.5 SO-PLS Prediction 33411.8.4.6 SO-PLS Validation 33411.8.4.7 Principal Components of Predictions 33611.8.4.8 CVANOVA 33611.8.5 Parallel and Orthogonalised Partial Least Squares 33711.8.5.1 PO-PLS Modelling 33711.8.5.2 PO-PLS Scores and Loadings 33811.8.6 Response Optimal Sequential Alternation 33911.8.6.1 ROSA Modelling 33911.8.6.2 ROSA Loadings 34011.8.6.3 ROSA Scores 34011.8.6.4 ROSA Prediction 34011.8.6.5 ROSA Validation 34111.8.6.6 ROSA Image Plots 34211.8.7 Multiblock Redundancy Analysis 34311.8.7.1 MB-RDA Modelling 34311.8.7.2 MB-RDA Loadings and Scores 34311.9 Complex Data Structures 34411.9.1 L-PLS 34411.9.1.1 Simulated L-shaped Data 34411.9.1.2 Exo-L-PLS 34411.9.1.3 Endo-L-PLS 34411.9.1.4 L-PLS Cross-validation 34511.9.2 SO-PLS-PM 34511.9.2.1 Single SO-PLS-PM Model 34611.9.2.2 Multiple Paths in an SO-PLS-PM Model 34611.10 Software Packages 34711.10.1 R Packages 34711.10.2 MATLAB Toolboxes 34811.10.3 Python 34911.10.4 Commercial Software 349                References 351Index 373

Age K. Smilde is a Professor of Biosystems Data Analysis at the Swammerdam Institute for Life Sciences at the University of Amsterdam. He also holds a part-time position at the Department of Machine Intelligence of Simula Metropolitan Center for Digital Engineering in Oslo, Norway. His research interest is multiblock data analysis and its implementation in different fields of life sciences. He is currently the Editor-in-Chief of the Journal of Chemometrics. Tormod Næs is a Senior Scientist at Nofima, a food research institute in Norway. He is also currently employed as adjoint professor at the Department of Food Science, University of Copenhagen, Denmark and as extraordinary professor at University of Stellenbosch, South Africa. His main research interest is multivariate analysis with special emphasis on applications in sensory science and spectroscopy. Kristian Hovde Liland is an Associate Professor with a top scientist scholarship in Data Science at the Norwegian University of Life Sciences and works in the areas of chemometrics, data analysis and machine learning. His main research is in linear prediction modelling, spectroscopy, and the transition between chemometrics and machine learning.

• Foreword xiiiPreface xvList of Figures xviiList of Tables xxxiPart I Introductory Concepts and Theory 11 Introduction 31.1 Scope of the Book 31.2 Potential Audience 41.3 Types of Data and Analyses 51.3.1 Supervised and Unsupervised Analyses 51.3.2 High-, Mid- and Low-level Fusion 51.3.3 Dimension Reduction 71.3.4 Indirect Versus Direct Data 81.3.5 Heterogeneous Fusion 81.4 Examples 81.4.1 Metabolomics 81.4.2 Genomics 111.4.3 Systems Biology 131.4.4 Chemistry 131.4.5 Sensory Science 151.5 Goals of Analyses 161.6 Some History 171.7 Fundamental Choices 171.8 Common and Distinct Components 191.9 Overview and Links 201.10 Notation and Terminology 211.11 Abbreviations 222 Basic Theory and Concepts 252.i General Introduction 252.1 Component Models 252.1.1 General Idea of Component Models 252.1.2 Principal Component Analysis 262.1.3 Sparse PCA 302.1.4 Principal Component Regression 312.1.5 Partial Least Squares 322.1.6 Sparse PLS 362.1.7 Principal Covariates Regression 372.1.8 Redundancy Analysis 382.1.9 Comparing PLS, PCovR and RDA 382.1.10 Generalised Canonical Correlation Analysis 382.1.11 Simultaneous Component Analysis 392.2 Properties of Data 392.2.1 Data Theory 392.2.2 Scale-types 422.3 Estimation Methods 442.3.1 Least-squares Estimation 442.3.2 Maximum-likelihood Estimation 452.3.3 Eigenvalue Decomposition-based Methods 472.3.4 Covariance or Correlation-based Estimation Methods 472.3.5 Sequential Versus Simultaneous Methods 482.3.6 Homogeneous Versus Heterogeneous Fusion 502.4 Within- and Between-block Variation 522.4.1 Definition and Example 522.4.2 MAXBET Solution 542.4.3 MAXNEAR Solution 542.4.4 PLS2 Solution 552.4.5 CCA Solution 552.4.6 Comparing the Solutions 562.4.7 PLS, RDA and CCA Revisited 562.5 Framework for Common and Distinct Components 602.6 Preprocessing 632.7 Validation 642.7.1 Outliers 642.7.1.1 Residuals 642.7.1.2 Leverage 662.7.2 Model Fit 672.7.3 Bias-variance Trade-off 692.7.4 Test Set Validation 702.7.5 Cross-validation 722.7.6 Permutation Testing 752.7.7 Jackknife and Bootstrap 762.7.8 Hyper-parameters and Penalties 772.8 Appendix 783 Structure of Multiblock Data 873.i General Introduction 873.1 Taxonomy 873.2 Skeleton of a Multiblock Data Set 873.2.1 Shared Sample Mode 883.2.2 Shared Variable Mode 883.2.3 Shared Variable or Sample Mode 883.2.4 Shared Variable and Sample Mode 893.3 Topology of a Multiblock Data Set 903.3.1 Unsupervised Analysis 903.3.2 Supervised Analysis 933.4 Linking Structures 953.4.1 Linking Structure for Unsupervised Analysis 953.4.2 Linking Structures for Supervised Analysis 963.5 Summary 984 Matrix Correlations 994.i General Introduction 994.1 Definition 994.2 Most Used Matrix Correlations 1014.2.1 Inner Product Correlation 1014.2.2 GCD coefficient 1014.2.3 RV-coefficient 1024.2.4 SMI-coefficient 1024.3 Generic Framework of Matrix Correlations 1044.4 Generalised Matrix Correlations 1054.4.1 Generalised RV-coefficient 1054.4.2 Generalised Association Coefficient 1064.5 Partial Matrix Correlations 1084.6 Conclusions and Recommendations 1104.7 Open Issues 111Part II Selected Methods for Unsupervised and Supervised Topologies 1135 Unsupervised Methods 1155.i General Introduction 1155.ii Relations to the General Framework 1155.1 Shared Variable Mode 1175.1.1 Only Common Variation 1175.1.1.1 Simultaneous Component Analysis 1175.1.1.2 Clustering and SCA 1235.1.1.3 Multigroup Data Analysis 1255.1.2 Common, Local, and Distinct Variation 1265.1.2.1 Distinct and Common Components 1275.1.2.2 Multivariate Curve Resolution 1305.2 Shared Sample Mode 1335.2.1 Only Common Variation 1335.2.1.1 SUM-PCA 1335.2.1.2 Multiple Factor Analysis and STATIS 1355.2.1.3 Generalised Canonical Analysis 1365.2.1.4 Regularised Generalised Canonical Correlation Analysis 1395.2.1.5 Exponential Family SCA 1405.2.1.6 Optimal-scaling 1435.2.2 Common, Local, and Distinct Variation 1465.2.2.1 Joint and Individual Variation Explained 1465.2.2.2 Distinct and Common Components 1475.2.2.3 PCA-GCA 1485.2.2.4 Advanced Coupled Matrix and Tensor Factorisation 1535.2.2.5 Penalised-ESCA 1565.2.2.6 Multivariate Curve Resolution 1585.3 Generic Framework 1595.3.1 Framework for Simultaneous Unsupervised Methods 1595.3.1.1 Description of the Framework 1595.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods 1615.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods 1615.4 Conclusions and Recommendations 1625.5 Open Issues 1646 ASCA and Extensions 1676.i General Introduction 1676.ii Relations to the General Framework 1676.1 ANOVA-Simultaneous Component Analysis 1686.1.1 The ASCA Method 1686.1.2 Validation of ASCA 1766.1.2.1 Permutation Testing 1766.1.2.2 Back-projection 1786.1.2.3 Confidence Ellipsoids 1786.1.3 The ASCA+ and LiMM-PCA Methods 1816.2 Multilevel-SCA 1826.3 Penalised-ASCA 1836.4 Conclusions and Recommendations 1856.5 Open Issues 1867 Supervised Methods 1877.i General Introduction 1877.ii Relations to the General Framework 1877.1 Multiblock Regression: General Perspectives 1887.1.1 Model and Assumptions 1887.1.2 Different Challenges and Aims 1887.2 Multiblock PLS Regression 1907.2.1 Standard Multiblock PLS Regression 1907.2.2 MB-PLS Used for Classification 1947.2.3 Sparse Multiblock PLS Regression (sMB-PLS) 1967.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression) 1997.3.1 The SO-PLS Method 1997.3.2 Order of Blocks 2027.3.3 Interpretation Tools 2027.3.4 Restricted PLS Components and their Application in SO-PLS 2037.3.5 Validation and Component Selection 2047.3.6 Relations to ANOVA 2057.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks 2127.3.8 Further Applications of SO-PLS 2157.3.9 Relations Between SO-PLS and ASCA 2157.4 Parallel and Orthogonalised PLS (PO-PLS) Regression 2177.5 Response Oriented Sequential Alternation 2227.5.1 The ROSA Method 2227.5.2 Validation 2257.5.3 Interpretation 2257.6 Conclusions and Recommendations 2287.7 Open Issues 229Part III Methods for Complex Multiblock Structures 2318 Complex Block Structures; with Focus on L-Shape Relations 2338.i General Introduction 2338.ii Relations to the General Framework 2348.1 Analysis of L-shape Data: General Perspectives 2358.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA 2368.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First 2368.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First 2388.2.3 Analysis of Segments/Clusters of X1 Data 2408.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data 2468.3.1 The Original L-PLS Method, Endo-L-PLS 2478.3.2 Exo- Versus Endo-L-PLS 2508.4 Modifications of the Original L-PLS Idea 2528.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter α2528.4.2 Three-blocks Bifocal PLS 2538.5 Alternative L-shape Data Analysis Methods 2548.5.1 Principal Component Analysis with External Information 2548.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration 2558.5.3 Multivariate Curve Resolution for Incomplete Data 2568.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1 2578.6 Domino PLS and More Complex Data Structures 2588.7 Conclusions and Recommendations 2588.8 Open Issues 260Part IV Alternative Methods for Unsupervised and Supervised Topologies 2619 Alternative Unsupervised Methods 2639.i General Introduction 2639.ii Relationship to the General Framework 2639.1 Shared Variable Mode 2639.2 Shared Sample Mode 2659.2.1 Only Common Variation 2659.2.1.1 DIABLO 2659.2.1.2 Generalised Coupled Tensor Factorisation 2669.2.1.3 Representation Matrices 2679.2.1.4 Extended PCA 2729.2.2 Common, Local, and Distinct Variation 2739.2.2.1 Generalised SVD 2739.2.2.2 Structural Learning and Integrative Decomposition 2739.2.2.3 Bayesian Inter-battery Factor Analysis 2759.2.2.4 Group Factor Analysis 2769.2.2.5 OnPLS 2779.2.2.6 Generalised Association Study 2789.2.2.7 Multi-Omics Factor Analysis 2789.3 Two Shared Modes and Only Common Variation 2819.3.1 Generalised Procrustes Analysis 2829.3.2 Three-way Methods 2829.4 Conclusions and Recommendations 2839.4.1 Open Issues 28410 Alternative Supervised Methods 28710.i General Introduction 28710.ii Relations to the General Framework 28710.1 Model and Focus 28810.2 Extension of PCovR 28810.2.1 Sparse Multiblock Principal Covariates Regression, Sparse PCovR 28810.2.2 Multiway Multiblock Covariates Regression 28910.3 Multiblock Redundancy Analysis 29210.3.1 Standard Multiblock Redundancy Analysis 29210.3.2 Sparse Multiblock Redundancy Analysis 29410.4 Miscellaneous Multiblock Regression Methods 29510.4.1 Multiblock Variance Partitioning 29610.4.2 Network Induced Supervised Learning 29610.4.3 Common Dimensions for Multiblock Regression 29810.5 Modifications and Extensions of the SO-PLS Method 29810.5.1 Extensions of SO-PLS to Three-Way Data 29810.5.2 Variable Selection for SO-PLS 29910.5.3 More Complicated Error Structure for SO-PLS 29910.5.4 SO-PLS Used for Path Modelling 30010.6 Methods for Data Sets Split Along the Sample Mode, Multigroup Methods 30410.6.1 Multigroup PLS Regression 30410.6.2 Clustering of Observations in Multiblock Regression 30610.6.3 Domain-Invariant PLS, DI-PLS 30710.7 Conclusions and Recommendations 30810.8 Open Issues 309Part V Software 31111 Algorithms and Software 31311.1 Multiblock Software 31311.2 R package multiblock 31311.3 Installing and Starting the Package 31411.4 Data Handling 31411.4.1 Read From File 31411.4.2 Data Pre-processing 31511.4.3 Re-coding Categorical Data 31611.4.4 Data Structures for Multiblock Analysis 31711.4.4.1 Create List of Blocks 31711.4.4.2 Create data.frame of Blocks 31711.5 Basic Methods 31811.5.1 Prepare Data 31911.5.2 Modelling 31911.5.3 Common Output Elements Across Methods 31911.5.4 Scores and Loadings 32011.6 Unsupervised Methods 32111.6.1 Formatting Data for Unsupervised Data Analysis 32111.6.2 Method Interfaces 32211.6.3 Shared Sample Mode Analyses 32211.6.4 Shared Variable Mode 32211.6.5 Common Output Elements Across Methods 32311.6.6 Scores and Loadings 32411.6.7 Plot From Imported Package 32511.7 ANOVA Simultaneous Component Analysis 32511.7.1 Formula Interface 32511.7.2 Simulated Data 32511.7.3 ASCA Modelling 32511.7.4 ASCA Scores 32611.7.5 ASCA Loadings 32611.8 Supervised Methods 32711.8.1 Formatting Data for Supervised Analyses 32711.8.2 Multiblock Partial Least Squares 32811.8.2.1 MB-PLS Modelling 32811.8.2.2 MB-PLS Summaries and Plotting 32811.8.3 Sparse Multiblock Partial Least Squares 32811.8.3.1 Sparse MB-PLS Modelling 32811.8.3.2 Sparse MB-PLS Plotting 32911.8.4 Sequential and Orthogonalised Partial Least Squares 33011.8.4.1 SO-PLS Modelling 33011.8.4.2 Måge Plot 33111.8.4.3 SO-PLS Loadings 33211.8.4.4 SO-PLS Scores 33311.8.4.5 SO-PLS Prediction 33411.8.4.6 SO-PLS Validation 33411.8.4.7 Principal Components of Predictions 33611.8.4.8 CVANOVA 33611.8.5 Parallel and Orthogonalised Partial Least Squares 33711.8.5.1 PO-PLS Modelling 33711.8.5.2 PO-PLS Scores and Loadings 33811.8.6 Response Optimal Sequential Alternation 33911.8.6.1 ROSA Modelling 33911.8.6.2 ROSA Loadings 34011.8.6.3 ROSA Scores 34011.8.6.4 ROSA Prediction 34011.8.6.5 ROSA Validation 34111.8.6.6 ROSA Image Plots 34211.8.7 Multiblock Redundancy Analysis 34311.8.7.1 MB-RDA Modelling 34311.8.7.2 MB-RDA Loadings and Scores 34311.9 Complex Data Structures 34411.9.1 L-PLS 34411.9.1.1 Simulated L-shaped Data 34411.9.1.2 Exo-L-PLS 34411.9.1.3 Endo-L-PLS 34411.9.1.4 L-PLS Cross-validation 34511.9.2 SO-PLS-PM 34511.9.2.1 Single SO-PLS-PM Model 34611.9.2.2 Multiple Paths in an SO-PLS-PM Model 34611.10 Software Packages 34711.10.1 R Packages 34711.10.2 MATLAB Toolboxes 34811.10.3 Python 34911.10.4 Commercial Software 349                References 351Index 373