Emotion Recognition: A Pattern Analysis Approach

A timely book containing foundations and current research directions on emotion recognition by facial expression, voice, gesture and biopotential signals This book provides a comprehensive examination of the research methodology of different modalities of emotion recognition. Key topics of discussion include facial expression, voice and biopotential signal-based emotion recognition. Special emphasis is given to feature selection, feature reduction, classifier design and multi-modal fusion to improve performance of emotion-classifiers. Written by several experts, the book includes several tools and techniques, including dynamic Bayesian networks, neural nets, hidden Markov model, rough sets, type-2 fuzzy sets, support vector machines and their applications in emotion recognition by different modalities. The book ends with a discussion on emotion recognition in automotive fields to determine stress and anger of the drivers, responsible for degradation of their performance and driving-ability. There is an increasing demand of emotion recognition in diverse fields, including psycho-therapy, bio-medicine and security in government, public and private agencies. The importance of emotion recognition has been given priority by industries including Hewlett Packard in the design and development of the next generation human-computer interface (HCI) systems. Emotion Recognition: A Pattern Analysis Approach would be of great interest to researchers, graduate students and practitioners, as the book Offers both foundations and advances on emotion recognition in a single volume Provides a thorough and insightful introduction to the subject by utilizing computational tools of diverse domains Inspires young researchers to prepare themselves for their own research Demonstrates direction of future research through new technologies, such as Microsoft Kinect, EEG systems etc.

Table of Contents:
Preface xix Acknowledgments xxvii Contributors xxix 1 Introduction to Emotion Recognition 1 Amit Konar, Anisha Halder, and Aruna Chakraborty 1.1 Basics of Pattern Recognition, 1 1.2 Emotion Detection as a Pattern Recognition Problem, 2 1.3 Feature Extraction, 3 1.4 Feature Reduction Techniques, 15 1.5 Emotion Classification, 17 1.6 Multimodal Emotion Recognition, 24 1.7 Stimulus Generation for Emotion Arousal, 24 1.8 Validation Techniques, 26 1.9 Summary, 27 References, 28 Author Biographies, 44 2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition 47 Yan Tong and Qiang Ji 2.1 Introduction, 48 2.2 Related Work, 49 2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN, 50 2.4 Experimental Results, 60 2.5 Conclusion, 64 References, 64 Author Biographies, 66 3 Facial Expressions: A Cross-Cultural Study 69 Chandrani Saha, Washef Ahmed, Soma Mitra, Debasis Mazumdar, and Sushmita Mitra 3.1 Introduction, 69 3.2 Extraction of Facial Regions and Ekman’s Action Units, 71 3.3 Cultural Variation in Occurrence of Different AUs, 76 3.4 Classification Performance Considering Cultural Variability, 79 3.5 Conclusion, 84 References, 84 Author Biographies, 86 4 A Subject-Dependent Facial Expression Recognition System 89 Chuan-Yu Chang and Yan-Chiang Huang 4.1 Introduction, 89 4.2 Proposed Method, 91 4.3 Experiment Result, 103 4.4 Conclusion, 109 Acknowledgment, 110 References, 110 Author Biographies, 112 5 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model 113 Md. Zia Uddin and Tae-Seong Kim 5.1 Introduction, 114 5.2 Methodology, 115 5.3 Experimental Results, 123 5.4 Conclusion, 125 Acknowledgments, 125 References, 126 Author Biographies, 127 6 Feature Selection for Facial Expression Based on Rough Set Theory 129 Yong Yang and Guoyin Wang 6.1 Introduction, 129 6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory, 131 6.3 Experiment Results and Discussion, 137 6.4 Conclusion, 143 Acknowledgments, 143 References, 143 Author Biographies, 145 7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets 147 Anisha Halder, Amit Konar, Aruna Chakraborty, and Atulya K. Nagar 7.1 Introduction, 148 7.2 Preliminaries on Type-2 Fuzzy Sets, 150 7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition, 152 7.4 Fuzzy Type-2 Membership Evaluation, 157 7.5 Experimental Details, 161 7.6 Performance Analysis, 167 7.7 Conclusion, 175 References, 176 Author Biographies, 180 8 Emotion Recognition from Non-frontal Facial Images 183 Wenming Zheng, Hao Tang, and Thomas S. Huang 8.1 Introduction, 184 8.2 A Brief Review of Automatic Emotional Expression Recognition, 187 8.3 Databases for Non-frontal Facial Emotion Recognition, 191 8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images, 196 8.5 Discussions and Conclusions, 205 Acknowledgments, 206 References, 206 Author Biographies, 211 9 Maximum a Posteriori Based Fusion Method for Speech Emotion Recognition 215 Ling Cen, Zhu Liang Yu, and Wee Ser 9.1 Introduction, 216 9.2 Acoustic Feature Extraction for Emotion Recognition, 219 9.3 Proposed Map-Based Fusion Method, 223 9.4 Experiment, 229 9.5 Conclusion, 232 References, 232 Author Biographies, 234 10 Emotion Recognition in Naturalistic Speech and Language—A Survey 237 Felix Weninger, Martin W¨ollmer, and Björn Schuller 10.1 Introduction, 238 10.2 Tasks and Applications, 239 10.3 Implementation and Evaluation, 244 10.4 Challenges, 253 10.5 Conclusion and Outlook, 257 Acknowledgment, 259 References, 259 Author Biographies, 267 11 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques 269 Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis 11.1 Introduction, 270 11.2 Brain Activity and Emotions, 271 11.3 EEG-ER Systems: An Overview, 272 11.4 Emotion Elicitation, 273 11.5 Advanced Signal Processing in EEG-ER, 275 11.6 Concluding Remarks and Future Directions, 287 References, 289 Author Biographies, 292 12 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT 295 M. Murugappan 12.1 Introduction, 296 12.2 Related Work, 297 12.3 Research Methodology, 299 12.4 Experimental Results and Discussions, 306 12.5 Conclusion, 310 12.6 Future Work, 310 Acknowledgments, 310 References, 310 Author Biography, 312 13 Toward Affective Brain–Computer Interface: Fundamentals and Analysis of EEG-Based Emotion Classification 315 Yuan-Pin Lin, Tzyy-Ping Jung, Yijun Wang, and Julie Onton 13.1 Introduction, 316 13.2 Materials and Methods, 323 13.3 Results and Discussion, 327 13.4 Conclusion, 332 13.5 Issues and Challenges Toward ABCIs, 332 Acknowledgments, 336 References, 336 Author Biographies, 340 14 Bodily Expression for Automatic Affect Recognition 343 Hatice Gunes, Caifeng Shan, Shizhi Chen, and YingLi Tian 14.1 Introduction, 344 14.2 Background and Related Work, 345 14.3 Creating a Database of Facial and Bodily Expressions: The FABO Database, 353 14.4 Automatic Recognition of Affect from Bodily Expressions, 356 14.5 Automatic Recognition of Bodily Expression Temporal Dynamics, 361 14.6 Discussion and Outlook, 367 14.7 Conclusions, 369 Acknowledgments, 370 References, 370 Author Biographies, 375 15 Building a Robust System for Multimodal Emotion Recognition 379 Johannes Wagner, Florian Lingenfelser, and Elisabeth André 15.1 Introduction, 380 15.2 Related Work, 381 15.3 The Callas Expressivity Corpus, 382 15.4 Methodology, 386 15.5 Multisensor Data Fusion, 390 15.6 Experiments, 395 15.7 Online Recognition System, 399 15.8 Conclusion, 403 Acknowledgment, 404 References, 404 Author Biographies, 410 16 Semantic Audiovisual Data Fusion for Automatic Emotion Recognition 411 Dragos Datcu and Leon J. M. Rothkrantz 16.1 Introduction, 412 16.2 Related Work, 413 16.3 Data Set Preparation, 416 16.4 Architecture, 418 16.5 Results, 431 16.6 Conclusion, 432 References, 432 Author Biographies, 434 17 A Multilevel Fusion Approach for Audiovisual Emotion Recognition 437 Girija Chetty, Michael Wagner, and Roland Goecke 17.1 Introduction, 437 17.2 Motivation and Background, 438 17.3 Facial Expression Quantification, 440 17.4 Experiment Design, 444 17.5 Experimental Results and Discussion, 450 17.6 Conclusion, 456 References, 456 Author Biographies, 459 18 From a Discrete Perspective of Emotions to Continuous, Dynamic, and Multimodal Affect Sensing 461 Isabelle Hupont, Sergio Ballano, Eva Cerezo, and Sandra Baldassarri 18.1 Introduction, 462 18.2 A Novel Method for Discrete Emotional Classification of Facial Images, 465 18.3 A 2D Emotional Space for Continuous and Dynamic Facial Affect Sensing, 469 18.4 Expansion to Multimodal Affect Sensing, 474 18.5 Building Tools That Care, 479 18.6 Concluding Remarks and Future Work, 486 Acknowledgments, 488 References, 488 Author Biographies, 491 19 Audiovisual Emotion Recognition Using Semi-Coupled Hidden Markov Model with State-Based Alignment Strategy 493 Chung-Hsien Wu, Jen-Chun Lin, and Wen-Li Wei 19.1 Introduction, 494 19.2 Feature Extraction, 495 19.3 Semi-Coupled Hidden Markov Model, 500 19.4 Experiments, 504 19.5 Conclusion, 508 References, 509 Author Biographies, 512 20 Emotion Recognition in Car Industry 515 Christos D. Katsis, George Rigas, Yorgos Goletsis, and Dimitrios I. Fotiadis 20.1 Introduction, 516 20.2 An Overview of Application for the Car Industry, 517 20.3 Modality-Based Categorization, 517 20.4 Emotion-Based Categorization, 520 20.5 Two Exemplar Cases, 523 20.6 Open Issues and Future Steps, 536 20.7 Conclusion, 537 References, 537 Author Biographies, 543 Index 545