Machine and Deep Learning Using MATLAB

Kamal I. M. Al-Malah received his PhD degree from Oregon State University in 1993. He served as a Professor of Chemical Engineering in Jordan and Gulf countries, as well as Former Chairman of the Chemical Engineering Department at the University of Hail in Saudi Arabia. Professor Al-Malah is an expert in both Aspen Plus® and MATLAB® applications. He has created a bundle of Windows-based software for engineering applications.

• Preface xiiiAbout the Companion Website xvii1 Unsupervised Machine Learning (ML) Techniques 1Introduction 1Selection of the Right Algorithm in ML 2Classical Multidimensional Scaling of Predictors Data 2Principal Component Analysis (PCA) 6k-Means Clustering 13Distance Metrics: Locations of Cluster Centroids 13Replications 14Gaussian Mixture Model (GMM) Clustering 15Optimum Number of GMM Clusters 17Observations and Clusters Visualization 18Evaluating Cluster Quality 21Silhouette Plots 22Hierarchical Clustering 23Step 1 -- Determine Hierarchical Structure 23Step 2 -- Divide Hierarchical Tree into Clusters 25PCA and Clustering: Wine Quality 27Feature Selection Using Laplacian (fsulaplacian) for Unsupervised Learning 35CHW 1.1 The Iris Flower Features Data 37CHW 1.2 The Ionosphere Data Features 38CHW 1.3 The Small Car Data 39CHW 1.4 Seeds Features Data 402 ML Supervised Learning: Classification Models 42Fitting Data Using Different Classification Models 42Customizing a Model 43Creating Training and Test Datasets 43Predicting the Response 45Evaluating the Classification Model 45KNN Model for All Categorical or All Numeric Data Type 47KNN Model: Heart Disease Numeric Data 48Viewing the Fitting Model Properties 50The Fitting Model: Number of Neighbors and Weighting Factor 51The Cost Penalty of the Fitting Model 52KNN Model: Red Wine Data 55Using MATLAB Classification Learner 57Binary Decision Tree Model for Multiclass Classification of All Data Types 68Classification Tree Model: Heart Disease Numeric Data Types 70Classification Tree Model: Heart Disease All Predictor Data Types 72Naive Bayes Classification Model for All Data Types 74Fitting Heart Disease Numeric Data to Naive Bayes Model 75Fitting Heart Disease All Data Types to Naive Bayes Model 77Discriminant Analysis (DA) Classifier for Numeric Predictors Only 79Discriminant Analysis (DA): Heart Disease Numeric Predictors 82Support Vector Machine (SVM) Classification Model for All Data Types 84Properties of SVM Model 85SVM Classification Model: Heart Disease Numeric Data Types 87SVM Classification Model: Heart Disease All Data Types 90Multiclass Support Vector Machine (fitcecoc) Model 92Multiclass Support Vector Machines Model: Red Wine Data 95Binary Linear Classifier (fitclinear) to High-Dimensional Data 98CHW 2.1 Mushroom Edibility Data 100CHW 2.2 1994 Adult Census Income Data 100CHW 2.3 White Wine Classification 101CHW 2.4 Cardiac Arrhythmia Data 102CHW 2.5 Breast Cancer Diagnosis 1023 Methods of Improving ML Predictive Models 103Accuracy and Robustness of Predictive Models 103Evaluating a Model: Cross-Validation 104Cross-Validation Tune-up Parameters 105Partition with K-Fold: Heart Disease Data Classification 106Reducing Predictors: Feature Transformation and Selection 108Factor Analysis 110Feature Transformation and Factor Analysis: Heart Disease Data 113Feature Selection 115Feature Selection Using predictorImportance Function: Health Disease Data 116Sequential Feature Selection (SFS): sequentialfs Function with Model Error Handler 118Accommodating Categorical Data: Creating Dummy Variables 121Feature Selection with Categorical Heart Disease Data 122Ensemble Learning 126Creating Ensembles: Heart Disease Data 130Ensemble Learning: Wine Quality Classification 131Improving fitcensemble Predictive Model: Abalone Age Prediction 132Improving fitctree Predictive Model with Feature Selection (FS): Credit Ratings Data 134Improving fitctree Predictive Model with Feature Transformation (FT): Credit Ratings Data 135Using MATLAB Regression Learner 136Feature Selection and Feature Transformation Using Regression Learner App 145Feature Selection Using Neighborhood Component Analysis (NCA) for Regression: Big Car Data 146CHW 3.1 The Ionosphere Data 148CHW 3.2 Sonar Dataset 149CHW 3.3 White Wine Classification 150CHW 3.4 Small Car Data (Regression Case) 1524 Methods of ML Linear Regression 153Introduction 153Linear Regression Models 154Fitting Linear Regression Models Using fitlm Function 155How to Organize the Data? 155Results Visualization: Big Car Data 162Fitting Linear Regression Models Using fitglm Function 164Nonparametric Regression Models 166fitrtree Nonparametric Regression Model: Big Car Data 167Support Vector Machine, fitrsvm, Nonparametric Regression Model: Big Car Data 170Nonparametric Regression Model: Gaussian Process Regression (GPR) 172Regularized Parametric Linear Regression 176Ridge Linear Regression: The Penalty Term 176Fitting Ridge Regression Models 177Predicting Response Using Ridge Regression Models 178Determining Ridge Regression Parameter, λ 179The Ridge Regression Model: Big Car Data 179The Ridge Regression Model with Optimum λ: Big Car Data 181Regularized Parametric Linear Regression Model: Lasso 183Stepwise Parametric Linear Regression 186Fitting Stepwise Linear Regression 187How to Specify stepwiselm Model? 187Stepwise Linear Regression Model: Big Car Data 188CHW 4.1 Boston House Price 192CHW 4.2 The Forest Fires Data 193CHW 4.3 The Parkinson’s Disease Telemonitoring Data 194CHW 4.4 The Car Fuel Economy Data 1955 Neural Networks 197Introduction 197Feed-Forward Neural Networks 198Feed-Forward Neural Network Classification 199Feed-Forward Neural Network Regression 200Numeric Data: Dummy Variables 200Neural Network Pattern Recognition (nprtool) Application 201Command-Based Feed-Forward Neural Network Classification: Heart Data 210Neural Network Regression (nftool) 214Command-Based Feed-Forward Neural Network Regression: Big Car Data 223Training the Neural Network Regression Model Using fitrnet Function: Big Car Data 226Finding the Optimum Regularization Strength for Neural Network Using Cross-Validation: Big Car Data 229Custom Hyperparameter Optimization in Neural Network Regression: Big Car Data 231CHW 5.1 Mushroom Edibility Data 233CHW 5.2 1994 Adult Census Income Data 233CHW 5.3 Breast Cancer Diagnosis 234CHW 5.4 Small Car Data (Regression Case) 234CHW 5.5 Boston House Price 2356 Pretrained Neural Networks: Transfer Learning 237Deep Learning: Image Networks 237Data Stores in MATLAB 241Image and Augmented Image Datastores 243Accessing an Image File 246Retraining: Transfer Learning for Image Recognition 247Convolutional Neural Network (CNN) Layers: Channels and Activations 256Convolution 2-D Layer Features via Activations 258Extraction and Visualization of Activations 261A 2-D (or 2-D Grouped) Convolutional Layer 264Features Extraction for Machine Learning 267Image Features in Pretrained Convolutional Neural Networks (CNNs) 268Classification with Machine Learning 268Feature Extraction for Machine Learning: Flowers 269Pattern Recognition Network Generation 271Machine Learning Feature Extraction: Spectrograms 275Network Object Prediction Explainers 278Occlusion Sensitivity 278imageLIME Features Explainer 282gradCAM Features Explainer 284HCW 6.1 CNN Retraining for Round Worms Alive or Dead Prediction 286HCW 6.2 CNN Retraining for Food Images Prediction 286HCW 6.3 CNN Retraining for Merchandise Data Prediction 287HCW 6.4 CNN Retraining for Musical Instrument Spectrograms Prediction 288HCW 6.5 CNN Retraining for Fruit/Vegetable Varieties Prediction 2897 A Convolutional Neural Network (CNN) Architecture and Training 290A Simple CNN Architecture: The Land Satellite Images 291Displaying Satellite Images 291Training Options 294Mini Batches 295Learning Rates 296Gradient Clipping 297Algorithms 298Training a CNN for Landcover Dataset 299Layers and Filters 302Filters in Convolution Layers 307Viewing Filters: AlexNet Filters 308Validation Data 311Using shuffle Function 316Improving Network Performance 319Training Algorithm Options 319Training Data 319Architecture 320Image Augmentation: The Flowers Dataset 322Directed Acyclic Graphs Networks 329Deep Network Designer (DND) 333Semantic Segmentation 342Analyze Training Data for Semantic Segmentation 343Create a Semantic Segmentation Network 345Train and Test the Semantic Segmentation Network 350HCW 7.1 CNN Creation for Round Worms Alive or Dead Prediction 356HCW 7.2 CNN Creation for Food Images Prediction 357HCW 7.3 CNN Creation for Merchandise Data Prediction 358HCW 7.4 CNN Creation for Musical Instrument Spectrograms Prediction 358HCW 7.5 CNN Creation for Chest X-ray Prediction 359HCW 7.6 Semantic Segmentation Network for CamVid Dataset 3598 Regression Classification: Object Detection 361Preparing Data for Regression 361Modification of CNN Architecture from Classification to Regression 361Root-Mean-Square Error 364AlexNet-Like CNN for Regression: Hand-Written Synthetic Digit Images 364A New CNN for Regression: Hand-Written Synthetic Digit Images 370Deep Network Designer (DND) for Regression 374Loading Image Data 375Generating Training Data 375Creating a Network Architecture 376Importing Data 378Training the Network 378Test Network 383YOLO Object Detectors 384Object Detection Using YOLO v4 386COCO-Based Creation of a Pretrained YOLO v4 Object Detector 387Fine-Tuning of a Pretrained YOLO v4 Object Detector 389Evaluating an Object Detector 394Object Detection Using R-CNN Algorithms 396R-CNN 397Fast R-CNN 397Faster R-CNN 398Transfer Learning (Re-Training) 399R-CNN Creation and Training 399Fast R-CNN Creation and Training 403Faster R-CNN Creation and Training 408evaluateDetectionPrecision Function for Precision Metric 413evaluateDetectionMissRate for Miss Rate Metric 417HCW 8.1 Testing yolov4ObjectDetector and fasterRCNN Object Detector 424HCW 8.2 Creation of Two CNN-based yolov4ObjectDetectors 424HCW 8.3 Creation of GoogleNet-Based Fast R-CNN Object Detector 425HCW 8.4 Creation of a GoogleNet-Based Faster R-CNN Object Detector 426HCW 8.5 Calculation of Average Precision and Miss Rate Using GoogleNet-Based Faster R-CNN Object Detector 427HCW 8.6 Calculation of Average Precision and Miss Rate Using GoogleNet-Based yolov4Object Detector 427HCW 8.7 Faster RCNN-based Car Objects Prediction and Calculation of Average Precision for Training and Test Data 4279 Recurrent Neural Network (RNN) 430Long Short-Term Memory (LSTM) and BiLSTM Network 430Train LSTM RNN Network for Sequence Classification 437Improving LSTM RNN Performance 441Sequence Length 441Classifying Categorical Sequences 445Sequence-to-Sequence Regression Using Deep Learning: Turbo Fan Data 446Classify Text Data Using Deep Learning: Factory Equipment Failure Text Analysis -- 1 453Classify Text Data Using Deep Learning: Factory Equipment Failure Text Analysis -- 2 462Word-by-Word Text Generation Using Deep Learning -- 1 465Word-by-Word Text Generation Using Deep Learning -- 2 473Train Network for Time Series Forecasting Using Deep Network Designer (DND) 475Train Network with Numeric Features 486HCW 9.1 Text Classification: Factory Equipment Failure Text Analysis 491HCW 9.2 Text Classification: Sentiment Labeled Sentences Data Set 492HCW 9.3 Text Classification: Netflix Titles Data Set 492HCW 9.4 Text Regression: Video Game Titles Data Set 492HCW 9.5 Multivariate Classification: Mill Data Set 493HCW 9.6 Word-by-Word Text Generation Using Deep Learning 49410 Image/Video-Based Apps 495Image Labeler (IL) App 495Creating ROI Labels 498Creating Scene Labels 499Label Ground Truth 500Export Labeled Ground Truth 501Video Labeler (VL) App: Ground Truth Data Creation, Training, and Prediction 502Ground Truth Labeler (GTL) App 513Running/Walking Classification with Video Clips using LSTM 520Experiment Manager (EM) App 526Image Batch Processor (IBP) App 533HCW 10.1 Cat Dog Video Labeling, Training, and Prediction -- 1 537HCW 10.2 Cat Dog Video Labeling, Training, and Prediction -- 2 537HCW 10.3 EM Hyperparameters of CNN Retraining for Merchandise Data Prediction 538HCW 10.4 EM Hyperparameters of CNN Retraining for Round Worms Alive or Dead Prediction 539HCW 10.5 EM Hyperparameters of CNN Retraining for Food Images Prediction 540Appendix A Useful MATLAB Functions 543A.1 Data Transfer from an External Source into MATLAB 543A.2 Data Import Wizard 543A.3 Table Operations 544A.4 Table Statistical Analysis 547A.5 Access to Table Variables (Column Titles) 547A.6 Merging Tables with Mixed Columns and Rows 547A.7 Data Plotting 548A.8 Data Normalization 549A.9 How to Scale Numeric Data Columns to Vary Between 0 and 1 549A.10 Random Split of a Matrix into a Training and Test Set 550A.11 Removal of NaN Values from a Matrix 550A.12 How to Calculate the Percent of Truly Judged Class Type Cases for a Binary Class Response 550A.13 Error Function m-file 551A.14 Conversion of Categorical into Numeric Dummy Matrix 552A.15 evaluateFit2 Function 553A.16 showActivationsForChannel Function 554A.17 upsampLowRes Function 555A.18A preprocessData function 555A.18B preprocessData2 function 555A.19 processTurboFanDataTrain function 556A.20 processTurboFanDataTest Function 556A.21 preprocessText Function 557A.22 documentGenerationDatastore Function 557A.23 subset Function for an Image Data Store Partition 560Index 561