Data-Variant Kernel Analysis

YUICHI MOTAI, Ph.D., is an Associate Professor of Electrical and Computer Engineering at the Virginia Commonwealth University, Richmond, Virginia. He received his Ph.D. with the Robot Vision Laboratory in the School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana in 2002.

• List of Figures xiiiList of Tables xixPreface xxiiiAcknowledgments xxvChapter 1 Survey 11.1 Introduction of Kernel Analysis 11.2 Kernel Offline Learning 21.2.1 Choose the Appropriate Kernels 31.2.2 Adopt KA into the Traditionally Developed Machine Learning Techniques 61.2.3 Structured Database with Kernel 91.3 Distributed Database with Kernel 121.3.1 Multiple Database Representation 121.3.2 Kernel Selections Among Heterogeneous Multiple Databases 131.3.3 Multiple Database Representation KA Applications to Distributed Databases 141.4 Kernel Online Learning 161.4.1 Kernel-Based Online Learning Algorithms 161.4.2 Adopt “Online” KA Framework into the Traditionally Developed Machine Learning Techniques 171.4.3 Relationship Between Online Learning and Prediction Techniques 211.5 Prediction with Kernels 221.5.1 Linear Prediction 221.5.2 Kalman Filter 231.5.3 Finite-State Model 231.5.4 Autoregressive Moving Average Model 241.5.5 Comparison of Four Models 251.6 Future Direction and Conclusion 26References 26Chapter 2 Offline Kernel Analysis 412.1 Introduction 412.2 Kernel Feature Analysis 432.2.1 Kernel Basics 432.2.2 Kernel Principal Component Analysis (KPCA) 452.2.3 Accelerated Kernel Feature Analysis (AKFA) 462.2.4 Comparison of the Relevant Kernel Methods 482.3 Principal Composite Kernel Feature Analysis (PC-KFA) 492.3.1 Kernel Selections 492.3.2 Kernel Combinatory Optimization 522.4 Experimental Analysis 542.4.1 Cancer Image Datasets 542.4.2 Kernel Selection 562.4.3 Kernel Combination and Reconstruction 582.4.4 Kernel Combination and Classification 592.4.5 Comparisons of Other Composite Kernel Learning Studies 602.4.6 Computation Time 612.5 Conclusion 61References 62Chapter 3 Group Kernel Feature Analysis 693.1 Introduction 693.2 Kernel Principal Component Analysis (KPCA) 713.3 Kernel Feature Analysis (KFA) for Distributed Databases 733.3.1 Extract Data-Dependent Kernels Using KFA 733.3.2 Decomposition of Database Through Data Association via Recursively Updating Kernel Matrices 753.4 Group Kernel Feature Analysis (GKFA) 783.4.1 Composite Kernel: Kernel Combinatory Optimization 793.4.2 Multiple Databases Using Composite Kernel 813.5 Experimental Results 833.5.1 Cancer Databases 833.5.2 Optimal Selection of Data-Dependent Kernels 843.5.3 Kernel Combinatory Optimization 843.5.4 Composite Kernel for Multiple Databases 863.5.5 K-NN Classification Evaluation with ROC 873.5.6 Comparison of Results with Other Studies on Colonography 893.5.7 Computational Speed and Scalability Evaluation of GKFA 903.6 Conclusions 91References 92Chapter 4 Online Kernel Analysis 974.1 Introduction 974.2 Kernel Basics: A Brief Review 994.2.1 Kernel Principal Component Analysis 994.2.2 Kernel Selection 1004.3 Kernel Adaptation Analysis of PC-KFA 1014.4 Heterogeneous vs. Homogeneous Data for Online PC-KFA 1024.4.1 Updating the Gram Matrix of the Online Data 1034.4.2 Composite Kernel for Online Data 1044.5 Long-Term Sequential Trajectories with Self-Monitoring 1044.5.1 Reevaluation of Large Online Data 1054.5.2 Validation of Decomposing Online Data into Small Chunks 1064.6 Experimental Results 1074.6.1 Cancer Datasets 1074.6.2 Selection of Optimum Kernel and Composite Kernel for Offline Data 1084.6.3 Selection of Optimum Kernel and Composite Kernel for the New Online Sequences 1104.6.4 Classification of Heterogeneous Versus Homogeneous Data 1114.6.5 Online Learning Evaluation of Long-term Sequence 1124.6.6 Evaluation of Computational Time 1164.7 Conclusions 117References 117Chapter 5 Cloud Kernel Analysis 1215.1 Introduction 1215.2 Cloud Environments 1235.2.1 Server Specifications of Cloud Platforms 1235.2.2 Cloud Framework of KPCA for AMD 1245.3 AMD for Cloud Colonography 1255.3.1 AMD Concept 1255.3.2 Data Configuration of AMD 1265.3.3 Implementation of AMD for Two Cloud Cases 1295.3.4 Parallelization of AMD 1325.4 Classification Evaluation of Cloud Colonography 1355.4.1 Databases with Classification Criteria 1355.4.2 Classification Results 1375.5 Cloud Computing Performance 1405.5.1 Cloud Computing Setting with Cancer Databases 1405.5.2 Computation Time 1425.5.3 Memory Usage 1445.5.4 Running Cost 1455.5.5 Parallelization 1455.6 Conclusions 146References 147Chapter 6 Predictive Kernel Analysis 1536.1 Introduction 1536.2 Kernel Basics 1546.2.1 KPCA and AKFA 1556.3 Stationary Data Training 1576.3.1 Kernel Selection 1576.3.2 Composite Kernel: Kernel Combinatory Optimization 1596.4 Longitudinal Nonstationary Data with Anomaly/Normal Detection 1606.4.1 Updating the Gram Matrix Based on Nonstationary Longitudinal Data 1606.4.2 Composite Kernel for Nonstationary Data 1626.5 Longitudinal Sequential Trajectories for Anomaly Detection and Prediction 1636.5.1 Anomaly Detection of Nonstationary Small Chunks Datasets 1646.5.2 Anomaly Prediction of Long-Time Sequential Trajectories 1676.6 Classification Results 1696.6.1 Cancer Datasets 1696.6.2 Selection of Optimum Kernel and Composite Kernel for Stationary Data 1706.6.3 Comparisons with Other Kernel Learning Methods 1726.6.4 Anomaly Detection for the Nonstationary Data 1746.7 Longitudinal Prediction Results 1756.7.1 Large Nonstationary Sequential dataset for Anomaly Detection 1756.7.2 Time Horizontal Prediction for Risk Factor Analysis of Anomaly Long-Time Sequential Trajectories 1786.7.3 Computational Time for Complexity Evaluation 1796.8 Conclusions 180References 181Chapter 7 Conclusion 185Appendix A 189Appendix B Representative Matlab codes 195B.1 Accelerated Kernel Feature Analysis 196B.2 Experimental Evaluations 198B.3 Group Kernel Analysis 201B.4 Online Composite Kernel Analysis 206B.5 Online Data Sequences Contol 208B.6 Alignment Factor 209B.7 Cloud Kernel Analysis 210B.8 Plot Computation Time 211B.9 Parallelization 212Index 215