About the Book
Language, vision and music: what common cognitive patterns underlie our competence in these disparate modes of thought? Language (natural & formal), vision and music seem to share at least the following attributes: a hierarchical organisation of constituents, recursivity, metaphor, the possibility of self-reference, ambiguity, and systematicity. Can we propose the existence of a general symbol system with instantiations in these three modes or is the only commonality to be found at the level of such entities as cerebral columnar automata? Answers are to be found in this international collection of work which recognises that one of the basic features of consciousness is its MultiModality, that there are possibilities to model this with contemporary technology, and that cross-cultural commonalities in the experience of, and creativity within, the various modalities are significant. With the advent of Intelligent MultiMedia this aspect of consciousness implementation in mind/brain acquires new significance. (Series B)
Table of Contents:
1. Dedication; 2. About the Editors; 3. Introduction (by Mc Kevitt, Paul); 4. Part I: Language & vision (by Mc Kevitt, Paul); 5. Multimedia integration: A system-theoretic perspective (by Connolly, John H.); 6. Visualising lexical prosodic representations for speech applications (by Carson-Berndsen, Julie); 7. A simulated language understanding agent using virtual perception (by Gurney, John); 8. The Hitchhiker's Guide to the Galaxy (by Cohen-Rose, A.L.); 9. Affective multimodal interaction with a 3D agent (by Brondsted, Tom); 10. CHAMELEON: A general platform for performing intellimedia (by Brondsted, Tom); 11. Machine perception of real-time multimodal natural dialogue (by Thorisson, Kris); 12. Communicative rhythm in gesture and speech (by Wachsmuth, Ipke); 13. Signals and meanings of gaze in animated faces (by Poggi, Isabella); 14. Speech, vision and aphasic communication (by Ahlsen, Elisabeth); 15. Synaesthesia and knowing (by Gammack, John G.); 16. What synaesthesia is (and is not) (by Day, Sean A.); 17. Synaesthesia is not a psychic anomaly, but a form of non-verbal thinking (by Galeyev, Bulat M.); 18. Part II: Language & music (by O Nuallain, Sean); 19. Music and language: Metaphor and causation (by Griffith, Niall J.L.); 20. Expression, content and meaning in language and music: An integrated semiotic analysis (by Callaghan, Jean); 21. Auditory structuring in explaining dyslexia (by Karma, Kai); 22. A comparative review of priming effects in language and music (by Tillmann, Barbara); 23. The respective roles of conscious- and subconscious processes for interpreting language and music (by Sabah, Gerard); 24. Aesthetic forms of expression as information delivery units (by Nemirovsky, Paul); 25. The lexicon of the Conductor's face (by Poggi, Isabella); 26. How do interactive virtual operas shift relationships between music, text and image? (by Bonardi, Alain); 27. "Let's Improvise Together": A testbed for a formalism in language vision and sounds integration (by Antonini, Riccardo); 28. On tonality in Irish traditional music (by O Nuallain, Sean); 29. The relationship between the imitation and recognition of non-verbal rhythms and language comprehension (by Treharne, Dilys); 30. Rising-falling contours in speech: A metaphor of tension-resolution schemes in European musical traditions? Evidence from regional varieties of Italian (by Romano, Antonio); 31. Part III: Creativity (by Mulvihill, Conn); 32. Plenary panel session: What is creativity? (by Antonini, Riccardo); 33. The analogical foundations of creativity in language, culture & the arts: "The Upper Paleolithic to 2100CE" (by Klein, Sheldon); 34. Creativity in humans, computers, and the rest of God's creatures: A meditation from within the economic world (by Rickards, Tudor); 35. The origins of Mexican metaphor in Tarahumara Indian religion (by Lonergan, Julia E.); 36. Is creativity algorithmic? (by Mulvihill, Conn); 37. Subject Index; 38. Author Index
Review :
Initiatives such as that represented here which bring together scholars from a wide range of cultural and linguistic backgrounds throw much light on natural language and alternative communicative systems and illuminate our understanding of the complex relationships that exist between language, mind and reality.
A vision on how modalities can get together to support creativity.
A guiding premise of this book is that an integrative approach to language, vision and music can inform us about the nature of both natural language and artificial communication systems and the complex interrelationships that exist between language, mind and machine. Contained in this volume are exciting examples of several sophisticated multimodal computer systems, architectures and interfaces, original experimental approaches relating language and music and some interesting work on the difficult topic of creativity.
A unique and diversified collection that takes an integrative approach to adress language, vision and music.
We have here an exciting international collection of work to reveal the similarities and differences between language, vision and music which will certainly create the next communicative systems.
