Emotion Recognition

Amit Konar is a Professor of Electronics and Tele-Communication Engineering, Jadavpur University, India, where he offers graduate-level courses on Artificial Intelligence and directs research in Cognitive Science, Robotics and Human-Computer Interfaces. Dr. Konar is the recipient of many prestigious grants and awards and is an author of 10 books and over 350 research publications. He offered consultancy services to Government and private industries. He served editorial services to many journals, including IEEE Transactions on Systems, Man and Cybernetics (Part-A) and IEEE Transactions on Fuzzy Systems.Aruna Chakraborty is an Associate Professor with the Department of Computer Science and Engineering, St. Thomas' College of Engineering and Technology, India. She is also a Visiting Faculty with Jadavpur University, where she offers graduate-level courses on Intelligent Automation and Robotics, and Cognitive Science. Her research interest includes human-computer interfaces, emotional intelligence and reasoning with fuzzy logic.

• Preface xixAcknowledgments xxviiContributors xxix1 Introduction to Emotion Recognition 1Amit Konar, Anisha Halder, and Aruna Chakraborty1.1 Basics of Pattern Recognition, 11.2 Emotion Detection as a Pattern Recognition Problem, 21.3 Feature Extraction, 31.4 Feature Reduction Techniques, 151.5 Emotion Classification, 171.6 Multimodal Emotion Recognition, 241.7 Stimulus Generation for Emotion Arousal, 241.8 Validation Techniques, 261.9 Summary, 27References, 28Author Biographies, 442 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition 47Yan Tong and Qiang Ji2.1 Introduction, 482.2 Related Work, 492.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN, 502.4 Experimental Results, 602.5 Conclusion, 64References, 64Author Biographies, 663 Facial Expressions: A Cross-Cultural Study 69Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita Mitra3.1 Introduction, 693.2 Extraction of Facial Regions and Ekman’s Action Units, 713.3 Cultural Variation in Occurrence of Different AUs, 763.4 Classification Performance Considering Cultural Variability, 793.5 Conclusion, 84References, 84Author Biographies, 864 A Subject-Dependent Facial Expression Recognition System 89Chuan-Yu Chang and Yan-Chiang Huang4.1 Introduction, 894.2 Proposed Method, 914.3 Experiment Result, 1034.4 Conclusion, 109Acknowledgment, 110References, 110Author Biographies, 1125 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model 113Md. Zia Uddin and Tae-Seong Kim5.1 Introduction, 1145.2 Methodology, 1155.3 Experimental Results, 1235.4 Conclusion, 125Acknowledgments, 125References, 126Author Biographies, 1276 Feature Selection for Facial Expression Based on Rough Set Theory 129Yong Yang and Guoyin Wang6.1 Introduction, 1296.2 Feature Selection for Emotion Recognition Based on Rough Set Theory, 1316.3 Experiment Results and Discussion, 1376.4 Conclusion, 143Acknowledgments, 143References, 143Author Biographies, 1457 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar7.1 Introduction, 1487.2 Preliminaries on Type-2 Fuzzy Sets, 1507.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 1527.4 Fuzzy Type-2 Membership Evaluation, 1577.5 Experimental Details, 1617.6 Performance Analysis, 1677.7 Conclusion, 175References, 176Author Biographies, 1808 Emotion Recognition from Non-frontal Facial Images 183Wenming Zheng, Hao Tang, and Thomas S. Huang8.1 Introduction, 1848.2 A Brief Review of Automatic Emotional Expression Recognition, 1878.3 Databases for Non-frontal Facial Emotion Recognition, 1918.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images, 1968.5 Discussions and Conclusions, 205Acknowledgments, 206References, 206Author Biographies, 2119 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition 215Ling Cen, Zhu Liang Yu, and Wee Ser9.1 Introduction, 2169.2 Acoustic Feature Extraction for Emotion Recognition, 2199.3 Proposed Map-Based Fusion Method, 2239.4 Experiment, 2299.5 Conclusion, 232References, 232Author Biographies, 23410 Emotion Recognition in Naturalistic Speech and Language—A Survey 237Felix Weninger, Martin W¨ollmer, and Björn Schuller10.1 Introduction, 23810.2 Tasks and Applications, 23910.3 Implementation and Evaluation, 24410.4 Challenges, 25310.5 Conclusion and Outlook, 257Acknowledgment, 259References, 259Author Biographies, 26711 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques 269Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis11.1 Introduction, 27011.2 Brain Activity and Emotions, 27111.3 EEG-ER Systems: An Overview, 27211.4 Emotion Elicitation, 27311.5 Advanced Signal Processing in EEG-ER, 27511.6 Concluding Remarks and Future Directions, 287References, 289Author Biographies, 29212 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT 295M. Murugappan12.1 Introduction, 29612.2 Related Work, 29712.3 Research Methodology, 29912.4 Experimental Results and Discussions, 30612.5 Conclusion, 31012.6 Future Work, 310Acknowledgments, 310References, 310Author Biography, 31213 Toward Affective Brain–Computer Interface: Fundamentals and Analysis of EEG-Based Emotion Classification 315Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton13.1 Introduction, 31613.2 Materials and Methods, 32313.3 Results and Discussion, 32713.4 Conclusion, 33213.5 Issues and Challenges Toward ABCIs, 332Acknowledgments, 336References, 336Author Biographies, 34014 Bodily Expression for Automatic Affect Recognition 343Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian14.1 Introduction, 34414.2 Background and Related Work, 34514.3 Creating a Database of Facial and Bodily Expressions: The FABO Database, 35314.4 Automatic Recognition of Affect from Bodily Expressions, 35614.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 36114.6 Discussion and Outlook, 36714.7 Conclusions, 369Acknowledgments, 370References, 370Author Biographies, 37515 Building a Robust System for Multimodal Emotion Recognition 379Johannes Wagner, Florian Lingenfelser, and Elisabeth André15.1 Introduction, 38015.2 Related Work, 38115.3 The Callas Expressivity Corpus, 38215.4 Methodology, 38615.5 Multisensor Data Fusion, 39015.6 Experiments, 39515.7 Online Recognition System, 39915.8 Conclusion, 403Acknowledgment, 404References, 404Author Biographies, 41016 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411Dragos Datcu and Leon J. M. Rothkrantz16.1 Introduction, 41216.2 Related Work, 41316.3 Data Set Preparation, 41616.4 Architecture, 41816.5 Results, 43116.6 Conclusion, 432References, 432Author Biographies, 43417 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437Girija Chetty, Michael Wagner, and Roland Goecke17.1 Introduction, 43717.2 Motivation and Background, 43817.3 Facial Expression Quantification, 44017.4 Experiment Design, 44417.5 Experimental Results and Discussion, 45017.6 Conclusion, 456References, 456Author Biographies, 45918 From a Discrete Perspective of Emotions to Continuous, Dynamic, and Multimodal Affect Sensing 461Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri18.1 Introduction, 46218.2 A Novel Method for Discrete Emotional Classification of Facial Images, 46518.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing, 46918.4 Expansion to Multimodal Affect Sensing, 47418.5 Building Tools That Care, 47918.6 Concluding Remarks and Future Work, 486Acknowledgments, 488References, 488Author Biographies, 49119 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy 493Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei19.1 Introduction, 49419.2 Feature Extraction, 49519.3 Semi-Coupled Hidden Markov Model, 50019.4 Experiments, 50419.5 Conclusion, 508References, 509Author Biographies, 51220 Emotion Recognition in Car Industry 515Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I. Fotiadis20.1 Introduction, 51620.2 An Overview of Application for the Car Industry, 51720.3 Modality-Based Categorization, 51720.4 Emotion-Based Categorization, 52020.5 Two Exemplar Cases, 52320.6 Open Issues and Future Steps, 53620.7 Conclusion, 537References, 537Author Biographies, 543Index 545