Multimodal Interactive Systems Management

This book provides a synthesis of the multifaceted field of interactive multimodal information management. The subjects treated include spoken language processing, image and video processing, document and handwriting analysis, identity information and interfaces. The book concludes with an overview of the highlights of the progress of the field during the past ten years, as well as the problems that are now under investigation and that offer the most promising results for the future. The book is addressed to the graduate student/postdoc level, but much of the book will be accessible to all those with a general background in information processing.

Table of Contents:
Interactive Multimodal Information Management: Shaping the Vision Meeting capture, analysis and access The IM2 Swiss National Center of Competence in Research Related international projects and consortia Human-Computer Interaction and Human Factors Human Factors in Multimodal Information Management Role of human factors Prominent research topics in human factors Methodological approach in human factors Empirical studies Discussion and implications User Attention During Mobile Video Consumption Modeling user behavior Data acquisition experiment Data processing and results Conclusions Wizard of Oz Evaluations of the Archivus Meeting Browser The Archivus meeting browser Multimodal Wizard of Oz evaluation Conclusions Document-Centric and Multimodal Meeting Assistants The Smart Meeting Minutes application Document centric meeting browsing Cross-meeting and ego-centric browsing Multimodal user interfaces prototyping for online meeting assistants The Communication Board application Conclusion Semantic Meeting Browsers and Assistants The JFerret framework and browser TQB: a transcript-based query and browsing interface Evaluation of meeting browsers Automatic meeting browsers and assistants Conclusions and perspectives Multimedia Information Retrieval Introduction Multimedia information retrieval: from information to user satisfaction Interaction log mining: from user satisfaction to improved information retrieval Multimedia information retrieval in a wider context Visual and Multimodal Analysis of Human Appearance and Behavior Face Recognition for Biometrics Introduction Face processing in a nutshell From face detection to face recognition Statistical generative models for face recognition Cross-pollination to other problems Open data and software Conclusion and future work Facial Expression Analysis Introduction and state-of-the-art Recognizing action units Modeling human perception of static facial expressions Conclusion Software for Automatic Gaze and Face/Object Tracking Gaze tracking Face tracking in real environments Application to autism spectrum disorder Conclusion Learning to Learn New Models of Human Activities in Indoor Settings Introduction Related work Proposed approach Activity tracking for unusual event detection Knowledge transfer for unusual event learning Experiments Conclusion Nonverbal Behavior Analysis Introduction: a brief history of nonverbal behavior research in IM2 VFOA recognition for communication analysis in meeting rooms and beyond Social signal processing Behavioral analysis of video blogging Final remarks Multimodal Biometric Person Recognition Introduction Biometric classification with quality measures Modeling reliability with Bayesian networks A-stack: biometric recognition in the score-age-quality classification space Conclusions Medical Image Annotation Introduction Multiple cues for image annotation Exploiting the hierarchical structure of data: confidence-based opinion fusion Facing the class imbalance problem: virtual examples Experiments Conclusions Speech, Language, and Document Processing Speech Processing Methods for automatic speech recognition Front-end processing of speech Posterior-based automatic speech recognition The Juicer decoder Conclusions Research Trends in Speaker Diarization Goals and applications of speaker diarization A state-of-the-art speaker diarization system Research problems in speaker diarization Conclusions and perspectives Speaker Diarization of Large Corpora Two-stage cross-meeting diarization Speaker linking Experimental results Conclusions Language Processing in Dialogues Objectives of language analysis in meetings Dialogue acts Discourse particles Thematic episodes and hot spots Semantic cross-modal alignment Conclusion and perspectives Offline Handwriting Recognition Introduction Offline word recognition From word to text recognition From text to documents Conclusions Online Handwriting Analysis and Recognition Introduction Database acquisition Online mode detection Online handwriting recognition Writer identification Conclusion Analysis of Printed Documents Extracting and reorganizing digital content from printable documents Tagging the information extracted from digital documents . . Video recording alignment and the temporal dimension of printable documents Aligning digital documents with audio recordings From printable documents to cross-media alignment and indexing Conclusion Assessments It was Worth it! Assessment of the Impact of IM2 Motivation and procedure The assessment questions Synthesis of the interviews Conclusion Technology Transfer: Turning Science into Products Visual recognition on mobile devices: kooaba AG Joining capture and webcast: Klewel SA Business experience meets technology: KeyLemon SA XED and Dolores for ebooks: sugarcube Information Technology Sarl From speech to text: Koemei SA Look me in the eye: Pomelo SARL The Association for Interactive Multimodal Information Management The International Create Challenge Conclusion and Perspectives Looking back on the initial motivations of IM2 Scientific achievements of IM2 Structural achievements Technology transfer achievements Perspectives