Unsupervised Learning

MATTHEW KYAN received his Ph.D. in Electrical Engineering in 2007 from the University of Sydney, Australia, winning the Siemens National Prize for Innovation for his work with 3-D confocal imaging. He is currently an Assistant Professor at Ryerson University, Toronto, Canada.PAISARN MUNEESAWANG received his Ph.D. from the school of Electrical and Information Engineering at the University of Sydney in 2002. He is currently an Associate Professor at Naresuan University.KAMBIZ JARRAH received his B.Eng. (with honors) in 2004 and M.A.Sc. in 2006, both in Electrical Engineering, from Ryerson University.LING GUAN is a Canada Research Chair in Multimedia and Computer Technology and a Professor in Electrical and Computer Engineering at Ryerson University, Canada.

• Acknowledgments xi 1 Introduction 11.1 Part I: The Self-Organizing Method 11.2 Part II: Dynamic Self-Organization for Image Filtering and Multimedia Retrieval 21.3 Part III: Dynamic Self-Organization for Image Segmentation and Visualization 51.4 Future Directions 72 Unsupervised Learning 92.1 Introduction 92.2 Unsupervised Clustering 92.3 Distance Metrics for Unsupervised Clustering 112.4 Unsupervised Learning Approaches 132.4.1 Partitioning and Cluster Membership 132.4.2 Iterative Mean-Squared Error Approaches 152.4.3 Mixture Decomposition Approaches 172.4.4 Agglomerative Hierarchical Approaches 182.4.5 Graph-Theoretic Approaches 202.4.6 Evolutionary Approaches 202.4.7 Neural Network Approaches 212.5 Assessing Cluster Quality and Validity 212.5.1 Cost Function–Based Cluster Validity Indices 222.5.2 Density-Based Cluster Validity Indices 232.5.3 Geometric-Based Cluster Validity Indices 243 Self-Organization 273.1 Introduction 273.2 Principles of Self-Organization 273.2.1 Synaptic Self-Amplification and Competition 273.2.2 Cooperation 283.2.3 Knowledge Through Redundancy 293.3 Fundamental Architectures 293.3.1 Adaptive Resonance Theory 293.3.2 Self-Organizing Map 373.4 Other Fixed Architectures for Self-Organization 433.4.1 Neural Gas 443.4.2 Hierarchical Feature Map 453.5 Emerging Architectures for Self-Organization 463.5.1 Dynamic Hierarchical Architectures 473.5.2 Nonstationary Architectures 483.5.3 Hybrid Architectures 503.6 Conclusion 504 Self-Organizing Tree Map 534.1 Introduction 534.2 Architecture 544.3 Competitive Learning 554.4 Algorithm 574.5 Evolution 614.5.1 Dynamic Topology 614.5.2 Classification Capability 644.6 Practical Considerations, Extensions, and Refinements 684.6.1 The Hierarchical Control Function 684.6.2 Learning, Timing, and Convergence 714.6.3 Feature Normalization 734.6.4 Stop Criteria 734.7 Conclusions 745 Self-Organization in Impulse Noise Removal 755.1 Introduction 755.2 Review of Traditional Median-Type Filters 765.3 The Noise-Exclusive Adaptive Filtering 825.3.1 Feature Selection and Impulse Detection 825.3.2 Noise Removal Filters 845.4 Experimental Results 865.5 Detection-Guided Restoration and Real-Time Processing 995.5.1 Introduction 995.5.2 Iterative Filtering 1015.5.3 Recursive Filtering 1045.5.4 Real-Time Processing of Impulse Corrupted TV Pictures 1055.5.5 Analysis of the Processing Time 1095.6 Conclusions 1156 Self-Organization in Image Retrieval 1196.1 Retrieval of Visual Information 1206.2 Visual Feature Descriptor 1226.2.1 Color Histogram and Color Moment Descriptors 1226.2.2 Wavelet Moment and Gabor Texture Descriptors 1236.2.3 Fourier and Moment-based Shape Descriptors 1256.2.4 Feature Normalization and Selection 1276.3 User-Assisted Retrieval 1306.3.1 Radial Basis Function Method 1326.4 Self-Organization for Pseudo Relevance Feedback 1366.5 Directed Self-Organization 1406.5.1 Algorithm 1426.6 Optimizing Self-Organization for Retrieval 1466.6.1 Genetic Principles 1476.6.2 System Architecture 1496.6.3 Genetic Algorithm for Feature Weight Detection 1506.7 Retrieval Performance 1536.7.1 Directed Self-Organization 1536.7.2 Genetic Algorithm Weight Detection 1556.8 Summary 1577 The Self-Organizing Hierarchical Variance Map 1597.1 An Intuitive Basis 1607.2 Model Formulation and Breakdown 1627.2.1 Topology Extraction via Competitive Hebbian Learning 1637.2.2 Local Variance via Hebbian Maximal Eigenfilters 1657.2.3 Global and Local Variance Interplay for Map Growth and Termination 1707.3 Algorithm 1737.3.1 Initialization, Continuation, and Presentation 1737.3.2 Updating Network Parameters 1757.3.3 Vigilance Evaluation and Map Growth 1757.3.4 Topology Adaptation 1767.3.5 Node Adaptation 1777.3.6 Optional Tuning Stage 1777.4 Simulations and Evaluation 1777.4.1 Observations of Evolution and Partitioning 1787.4.2 Visual Comparisons with Popular Mean-Squared Error Architectures 1817.4.3 Visual Comparison Against Growing Neural Gas 1837.4.4 Comparing Hierarchical with Tree-Based Methods 1837.5 Tests on Self-Determination and the Optional Tuning Stage 1877.6 Cluster Validity Analysis on Synthetic and UCI Data 1877.6.1 Performance vs. Popular Clustering Methods 1907.6.2 IRIS Dataset 1927.6.3 WINE Dataset 1957.7 Summary 1958 Microbiological Image Analysis Using Self-Organization 1978.1 Image Analysis in the Biosciences 1978.1.1 Segmentation: The Common Denominator 1988.1.2 Semi-supervised versus Unsupervised Analysis 1998.1.3 Confocal Microscopy and Its Modalities 2008.2 Image Analysis Tasks Considered 2028.2.1 Visualising Chromosomes During Mitosis 2028.2.2 Segmenting Heterogeneous Biofilms 2048.3 Microbiological Image Segmentation 2058.3.1 Effects of Feature Space Definition 2078.3.2 Fixed Weighting of Feature Space 2098.3.3 Dynamic Feature Fusion During Learning 2138.4 Image Segmentation Using Hierarchical Self-Organization 2158.4.1 Gray-Level Segmentation of Chromosomes 2158.4.2 Automated Multilevel Thresholding of Biofilm 2208.4.3 Multidimensional Feature Segmentation 2218.5 Harvesting Topologies to Facilitate Visualization 2268.5.1 Topology Aware Opacity and Gray-Level Assignment 2278.5.2 Visualization of Chromosomes During Mitosis 2288.6 Summary 2339 Closing Remarks and Future Directions 2379.1 Summary of Main Findings 2379.1.1 Dynamic Self-Organization: Effective Models for Efficient Feature Space Parsing 2379.1.2 Improved Stability, Integrity, and Efficiency 2389.1.3 Adaptive Topologies Promote Consistency and Uncover Relationships 2399.1.4 Online Selection of Class Number 2399.1.5 Topologies Represent a Useful Backbone for Visualization or Analysis 2409.2 Future Directions 2409.2.1 Dynamic Navigation for Information Repositories 2419.2.2 Interactive Knowledge-Assisted Visualization 2439.2.3 Temporal Data Analysis Using Trajectories 245Appendix A 249A.1 Global and Local Consistency Error 249References 251Index 269