Conversational Informatics – An Engineering Approach

Conversational informatics investigates human behaviour with a view to designing conversational artifacts capable of interacting with humans in a conversational fashion. It spans a broad array of topics including linguistics, psychology and human-computer interaction. Until recently research in such areas has been carried out in isolation, with no attempt made to connect the various disciplines. Advancements in science and technology have changed this. Conversational Informatics provides an interdisciplinary introduction to conversational informatics and places emphasis upon the integration of scientific approaches to achieve engineering goals and to advance further understanding of conversation. It features a collection of surveys structured around four prominent research areas: conversational artifacts, conversational contents, conversation environment design and conversation measurement, analysis and modelling *Conversational artifacts shows how synthetic characters or intelligent robots use eye gaze, gestures and other non-verbal communicators to interact.* Conversational contents looks at developing techniques for acquiring, editing, distributing and utilising the contents that are produced and consumed in conversation.*Conversation environment design explains techniques for creating intelligent virtual environments and for representing individuals within a virtual environment by monitoring and reproducing their non-verbal conversational behaviour.*Conversation measurement, analysis and modelling demonstrate how conversational behaviour can be measured and analyzed. Conversational Informatics will be an invaluable resource for postgraduate students and researchers in Computer Science and Electrical Engineering as well as engineers and developers working in the field of automation, robotics and agents technology.

Table of Contents:
Preface. List of Contributors. 1 Introduction (Toyoaki Nishida). 1.1 Conversation: the Most Natural Means of Communication. 1.2 An Engineering Approach to Conversation. 1.3 Towards a Breakthrough. 1.4 Approaches Used in Conversational Informatics. 1.5 Conversational Artifacts. 1.6 Conversational Content. 1.7 Conversational Environment Design. 1.8 Conversation Measurement, Analysis, and Modeling. 1.9 Underlying Methodology. References. Part I Conversational Artifacts. 2 Conversational Agents and the Construction of Humorous Acts (Anton Nijholt). 2.1 Introduction. 2.2 The Role of Humor in Interpersonal Interaction. 2.3 Embodied Conversation Agents. 2.4 Appropriateness of Humorous Acts in Conversations. 2.5 Humorous Acts and Computational Humor. 2.6 Nonverbal Support for Humorous Acts. 2.7 Methods, Tools, Corpora, and Future Research. 2.8 Conclusions. References. 3 Why Emotions should be Integrated into Conversational Agents (Christian Becker, Stefan Kopp, and Ipke Wachsmuth). 3.1 Introduction and Motivation. 3.2 How to Conceptualize Emotions. 3.3 Why to Integrate Emotions into Conversational Agents. 3.4 Making the Virtual Human Max Emotional. 3.5 Examples and Experiences. 3.6 Conclusions. References. 4 More Than Just a Friendly Phrase: Multimodal Aspects of Polite Behavior in Agents (Matthias Rehm and Elisabeth Andre) 4.1 Introduction. 4.2 The Augsburg SEMMEL Corpus. 4.3 Employing the Results for ECA Control. 4.4 Evaluating Multimodal Politeness Behavior. 4.5 Conclusions. References. 5 Attentional Behaviors as Nonverbal Communicative Signals in Situated Interactions with Conversational Agents (Yukiko I. Nakano and Toyoaki Nishida). 5.1 Introduction. 5.2 Related Work. 5.3 Nonverbal Grounding using Attentional Behaviors Towards the Physical World. 5.4 Dialogue Management using Attentional Behaviors Towards the Virtual World. 5.5 Conclusions. References. 6 Attentional Gestures in Dialogues Between People and Robots (Candace L. Sidner and Christopher Lee). 6.1 Introduction. 6.2 Background and Related Research. 6.3 A Conversational Robot. 6.4 Looking Behaviors for the Robot. 6.5 Nodding at the Robot. 6.6 Lessons Learned. 6.7 Future Directions. References. 7 Dialogue Context for Visual Feedback Recognition (Louis-Philippe Morency, Candace L. Sidner, and Trevor Darrell). 7.1 Introduction. 7.2 Background and Related Research. 7.3 Context for Visual Feedback. 7.4 Context from Dialogue Manager. 7.5 Framework for Context-based Gesture Recognition. 7.6 Contextual Features. 7.7 Context-based Head Gesture Recognition. 7.8 Conclusions. References. 8 Trading Spaces: How Humans and Humanoids Use Speech and Gesture to Give Directions (Stefan Kopp, Paul A. Tepper, Kimberley Ferriman, Kristina Striegnitz, and Justine Cassell). 8.1 Introduction. 8.2 Words and Gestures for Giving Directions. 8.3 Relationship between Form and Meaning of Iconic Gestures in Direction-giving. 8.4 Discussion of Empirical Results. 8.5 Generating Directions with Humanoids. 8.6 Multimodal Microplanning. 8.7 Surface Realization. 8.8 Discussion of Generation Results. 8.9 Conclusions. References. 9 Facial Gestures: Taxonomy and Application of Nonverbal, Nonemotional Facial Displays for Embodied Conversational Agents (Goranka Zoric, Karlo Smid, and Igor S. Pandzic). 9.1 Introduction. 9.2 Facial Gestures for Embodied Conversational Agents. 9.3 Example of a Practical System Implementation. 9.4 Results. 9.5 Conclusions. References. Part II Conversational Contents. 10 Conversation Quantization and Sustainable Knowledge Globe (Hidekazu Kubota, Yasuyuki Sumi, and Toyoaki Nishida). 10.1 Introduction. 10.2 Conversation Quantization. 10.3 Knowledge Landscape. 10.4 Experiments. 10.5 Discussion. 10.6 Conclusions. References. 11 Automatic Text Presentation for the Conversational Knowledge Process (Sadao Kurohashi, Daisuke Kawahara, Nobuhiro Kaji, and Tomohide Shibata). 11.1 Introduction. 11.2 Current State of Natural Language Processing. 11.3 Unit of Conversation: the Knowledge Card. 11.4 Paraphrasing Written Language to Spoken Language. 11.5 Automatic Slide Generation. 11.7 Experiments and Discussion. 11.6 Conclusions. References. 12 Video Content Acquisition and Editing for Conversation Scenes (Yuichi Nakamura). 12.1 Introduction. 12.2 Obtaining Conversation Contents. 12.3 Capturing Conversation Scenes. 12.4 Editing Conversation Scenes. 12.5 Example of Capture and Editing. 12.6 Performance Evaluation. 12.7 Conclusion. References. 13 Personalization of Video Contents (Noboru Babaguchi). 13.1 Introduction. 13.2 Related Work. 13.3 Metadata. 13.4 Profile. 13.5 Definition of Video Summarization. 13.6 Scene Significance. 13.7 Generation of Video Digest. 13.8 Generation of Video Poster. 13.9 Experimental Results. 13.10 Discussion. 13.11 Conclusions. References. Part III Conversational Environment Design. 14 Conversational Content Acquisition by Ubiquitous Sensors (Yasuyuki Sumi, Kenji Mase, and Toyoaki Nishida). 14.1 Introduction. 14.2 Capturing Conversation Scenes by Multiple Sensors. 14.3 Segmentation and Interpretation of Scenes. 14.4 Video Summary: Chronological Collage of Multiple-viewpoint Videos. 14.5 Building 3D Virtual Space by Spatiotemporal Video Collage. 14.6 The Ambient Sound Shower: Sound Collage for Revealing Situated Conversations. 14.7 Inferring Semantic Information about Detected Conversation Scenes by Nonverbal Information. 14.8 Related Work. 14.9 Conclusions. References. 15 Real-time Human Proxy (Rin-ichiro Taniguchi and Daisaku Arita). 15.1 Introduction. 15.2 Concept of Real-time Human Proxy. 15.3 Acquisition of Human Motion. 15.4 Presentation of Avatar. 15.5 Prototype of Real-time Human Proxy. 15.6 Conclusions. References. 16 Lecture Archiving System (Satoshi Nishiguchi, Koh Kakusho, and Michihiko Minoh). 16.1 Introduction. 16.2 Concept of Environmental Media. 16.3 Related Works. 16.4 Definition of Dynamic Situations in the Classroom. 16.5 Recognition of Dynamic Situations. 16.6 Speaker Detection by Multimodal Sensors. 16.7 Experimental Results. 16.8 Conclusions. References. Part IV Conversational Measurement, Analysis, and Modeling. 17 A Scientific Approach to Conversational Informatics: Description, Analysis, and Modeling of Human Conversation (Yasuharu Den and Mika Enomoto). 17.1 Introduction. 17.2 Recording and Description of Multimodal Three-party Conversations. 17.3 Analysis of Multimodal Three-Party Conversations. 17.4 Modeling Human-to-Human Conversation. 17.5 Conclusions. References. 18 Embodied Synchrony in Conversation (Chika Nagaoka, Masashi Komori, and Sakiko Yoshikawa). 18.1 Introduction. 18.2 Measurement and Quantification of Embodied Synchrony. 18.3 Influences on Embodied Synchrony. 18.4 Embodied Synchrony and Human Behavior. 18.5 Model and Theory of Embodied Synchrony. 18.6 Conclusions. References. 19 Modeling Communication Atmosphere (Tomasz M. Rutkowski and Danilo P. Mandic). 19.1 Introduction. 19.2 Communication Atmosphere. 19.3 Automatic Assessment of Communication Atmosphere: Machine Learning Methods. 19.4 Experiments. 19.5 Conclusions. References. 20 Analysis of Interaction Mechanisms in Online Communities (Naohiro Matsumura). 20.1 Introduction. 20.2 Four Roles for Individuals. 20.3 Measuring the Influence of Individuals. 20.4 Three Types of Communication. 20.5 Frequent Communication Patterns. 20.6 Conclusions. References. 21 Mutual Adaptation: A New Criterion for Designing and Evaluating Human-Computer Interaction (Kazuhiro Ueda and Takanori Komatsu). 21.1 Introduction. 21.2 Communication Experiment. 21.3 Proposal for a Meaning-acquisition Model. 21.4 Interaction between the Proposed Model and Users. 21.5 Discussion. 21.6 Conclusions. References. Index.