Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immersive Telepresence using SIP: Scalable Distributed Applications and Media Control over Internet

Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immsersive Telepresence using SIP: Scalable Distributed Applications and Media Control over Internet is the first book to put together all IETF request for comments (RFCs), and the internet drafts standards related to the multipoint conferencing and immersive telepresence. This book includes mandatory and optional texts of all standards in a chronological and systematic way almost with one-to-one integrity from the beginning to end, allowing the reader to understand all aspects of the highly complex real-time applications. It is a book that network designers, software developers, product manufacturers, implementers, interoperability testers, professionals, professors, and researchers will find to be immensely useful. Practitioners and engineers in all spectrums who are concentrating on building the real-time, scalable, interoperable multipoint applications, can use this book to make informed choices based on technical standards in the market place, on all proprietary non-scalable and non-interposable products. This book will provide focus and foundation for these decision makers.

Table of Contents:
Contents Preface...........................................................................................................................................xxxi Author...........................................................................................................................................xxxv Chapter 1 Scalable Networked Multipoint Multimedia Conferencing and Telepresence..............1 1.1 Networked Multipoint Multimedia Conferencing..............................................1 1.2 Basic Conferencing Protocol Architecture.........................................................4 1.3 Third-Party Multiparty Conferencing.............................................................. 12 1.4 Multistream Immersive Telepresence...............................................................23 1.5 Scalable Centralized Multipoint Multimedia Conferencing Systems and Protocols Framework.................................................................................24 1.6 Terminology.....................................................................................................28 1.7 Summary..........................................................................................................28 1.8 Problems........................................................................................................... 41 References................................................................................................................... 42 Chapter 2 Centralized Conferencing System Architecture......................................................... 43 2.1 Introduction...................................................................................................... 43 2.2 Convention........................................................................................................ 43 2.3 Terminology..................................................................................................... 43 2.4 Overview..........................................................................................................44 2.5 Centralized Conferencing Data........................................................................ 45 2.6 Centralized Conferencing Constructs and Identifiers...................................... 47 2.7 Conferencing System Realization....................................................................50 2.8 Conferencing Mechanisms............................................................................... 55 2.9 Conference Scenario Realizations....................................................................56 2.10 Relationships between SIP and Centralized Conferencing System Frameworks......................................................................................................69 2.11 Security Considerations....................................................................................69 2.12 Summary.......................................................................................................... 71 2.13 Problems........................................................................................................... 72 Chapter 3 Media Server Control Architecture............................................................................. 73 3.1 Introduction...................................................................................................... 73 3.2 Terminology..................................................................................................... 75 3.3 Architecture Overview..................................................................................... 75 3.4 NSIP Usage.......................................................................................................77 3.5 Media Control for IVR Services...................................................................... 78 3.6 Media Control for Conferencing Services........................................................80 3.7 Security Considerations....................................................................................85 3.8 Summary..........................................................................................................86 3.9 Problems...........................................................................................................87 Chapter 4 Conferencing Information Data Model for Centralized Conferencing.......................89 4.1 Introduction......................................................................................................89 4.2 Terminology.....................................................................................................89 4.3 Overview..........................................................................................................89 Comparison.......................................................................... 91 4.4 Data Model Definition......................................................................................92 4.5 RELAX NG Schema...................................................................................... 103 4.6 XML Schema Extensibility............................................................................ 103 4.7 XML Example................................................................................................ 103 4.8 Non-Normative RELAX NG Schema in XML Syntax (Appendix A of RFC 6501)....................................................................................................... 103 4.9 Relax Schema in XML Syntax (Appendix B of RFC 6501).......................... 103 4.10 Security Considerations.................................................................................. 104 4.11 Summary........................................................................................................ 108 4.12 Problems......................................................................................................... 109 References................................................................................................................. 111 Chapter 5 Centralized Conferencing Manipulation Protocol.................................................... 129 5.1 Introduction.................................................................................................... 129 5.2 Conventions and Terminology........................................................................ 129 5.3 XCON Conference Control System Architecture.......................................... 129 5.4 Protocol Overview.......................................................................................... 131 5.5 CCMP Messages............................................................................................ 134 5.6 Example of CCMP in Action......................................................................... 153 5.7 Locating a Conference Server........................................................................ 162 5.8 Managing Notifications.................................................................................. 163 5.9 HTTP Transport............................................................................................. 165 5.10 Security Considerations.................................................................................. 166 5.11 XML Schema................................................................................................. 168 5.12 IANA Considerations..................................................................................... 169 5.13 Summary........................................................................................................ 176 5.14 Evaluation of Other Protocol Models and Transports Considered for CCMP (Appendix A of RFC 6503)................................................................ 176 5.15 Problems......................................................................................................... 178 References................................................................................................................. 179 Chapter 6 Binary Floor Control Protocol.................................................................................. 181 6.1 Introduction.................................................................................................... 181 6.2 Terminology................................................................................................... 181 6.3 Scope.............................................................................................................. 181 6.4 Overview of Operation................................................................................... 183 6.5 Packet Format................................................................................................. 186 6.6 Transport........................................................................................................200 6.7 Lower-Layer Security..................................................................................... 201 6.8 Protocol Transactions..................................................................................... 201 6.9 Authentication and Authorization..................................................................202 6.10 Floor Participant Operations..........................................................................202 6.11 Chair Operations............................................................................................205 6.12.1 Requesting Information about Floors...............................................206 6.13 Floor Control Server Operations....................................................................208 6.14 Security Considerations.................................................................................. 214 6.15 BFCP Connection Establishment................................................................... 215 6.16 IANA Considerations..................................................................................... 218 6.17 Summary........................................................................................................ 221 6.18 Problems.........................................................................................................222 Chapter 7 XCON Notification Service.......................................................................................225 7.1 Introduction....................................................................................................225 7.2 Terminology...................................................................................................225 7.3 Notification Formats.......................................................................................225 7.4 Full Notifications............................................................................................225 7.5 Partial Notifications........................................................................................227 7.6 IANA Considerations.....................................................................................229 7.7 Security Considerations..................................................................................229 7.7.1 Connection Security..........................................................................230 7.7.2 Authorization Considerations ...........................................................230 7.8 Summary........................................................................................................230 7.9 Problems......................................................................................................... 231 Reference................................................................................................................... 231 Chapter 8 Media Channel Control Framework.......................................................................... 233 8.1 Introduction.................................................................................................... 233 8.2 CFW Capabilities........................................................................................... 233 8.3 Overview........................................................................................................ 235 8.4 Control Channel Setup................................................................................... 238 8.4.1 Control Client SIP UAC Behavior..................................................... 238 8.4.2 Control Server SIP UAS Behavior.................................................... 239 8.5 Establishing Media Streams – Control Client SIP UAC Behavior.................240 8.6 Control Framework Interactions..................................................................... 241 8.7 Response Code Descriptions.......................................................................... 247 8.8 Control Packages............................................................................................ 247 8.9 Formal Syntax................................................................................................249 8.10 Examples........................................................................................................ 251 8.11 Extensibility....................................................................................................254 8.12 Security Considerations..................................................................................254 8.12.1 Session Establishment.......................................................................254 8.13 IANA Considerations.....................................................................................256 8.14 Common Package Components (Appendix A of RFC 6230).........................256 8.15 Summary........................................................................................................ 257 8.16 Problems......................................................................................................... 257 Chapter 9 Mixer Package for Media Control Channel Framework........................................... 259 9.1 Introduction.................................................................................................... 259 9.2 Conventions and Terminology........................................................................260 9.3 Control Package Definition............................................................................260 9.4 Element Definitions........................................................................................ 262 9.5 Formal Syntax................................................................................................290 9.6 Examples........................................................................................................ 291 9.6.1 AS-MS Framework Interaction Examples........................................ 291 9.6.1.1 Creating a Conference Mixer and Joining a Participant..........................................................................309 9.6.1.2 Receiving Active Talker Notifications...............................309 9.6.1.3 Conference Termination....................................................309 9.6.2 Mixing Examples.............................................................................. 310 9.6.2.1 Audio Conferencing........................................................... 310 9.6.2.2 Bridging Connections........................................................ 312 xvi Contents 9.6.2.3 Video Conferencing........................................................... 312 9.7 Security Considerations.................................................................................. 313 9.8 IANA Considerations..................................................................................... 315 9.9 Summary........................................................................................................ 315 9.10 Problems......................................................................................................... 315 References................................................................................................................. 316 Chapter 10 Media Session Recording.......................................................................................... 317 10.1 Use Cases and Requirements for SIP-Based Media Recording................... 317 10.2 Media Recording Architecture....................................................................324 10.3 Media Recording Metadata.......................................................................... 331 XML Body.................................................................344 10.3.9 XML Schema Definition for Recording Metadata....................... 345 10.3.10 Security Considerations................................................................348 10.3.11 IANA Considerations...................................................................349 10.4 Media Session Recoding Protocol...............................................................349 10.5 Summary...................................................................................................... 371 10.6 Problems....................................................................................................... 372 References................................................................................................................. 373 Chapter 11 Media Resource Brokering....................................................................................... 375 11.1 Introduction.................................................................................................. 375 11.2 Conventions and Terminology..................................................................... 375 11.3 Problem Discussion...................................................................................... 377 11.4 Deployment Scenario Options..................................................................... 378 11.5 MRB Interface Definitions.......................................................................... 381 11.6 MRB Acting as a B2BUA............................................................................405 11.7 Multimodal MRB Implementations.............................................................405 11.8 Relative Merits of Query Mode, IAMM, and IUMM.................................406 11.9 Examples......................................................................................................407 11.10 Media Service Resource Publisher Interface XML Schema....................... 425 11.11 Media Service Resource Consumer Interface XML Schema......................444 11.12 Security Considerations............................................................................... 461 11.13 IANA Considerations...................................................................................463 11.14 Summary......................................................................................................463 11.15 Problems.......................................................................................................464 References.................................................................................................................465 Chapter 12 Media Control Channel Framework for Interactive Voice Response ....................... 467 12.1 Introduction.................................................................................................. 467 12.2 Conventions and Terminology.....................................................................469 12.3 Control Package Definition..........................................................................469 12.4 Element Definitions...................................................................................... 471 12.5 Formal Syntax.............................................................................................. 511 12.6 Examples...................................................................................................... 534 12.8 IANA Considerations...................................................................................544 12.9 Using VoiceXML as a Dialog Language.....................................................544 12.10 Summary...................................................................................................... 549 12.11 Problems....................................................................................................... 550 Chapter 13 SIP Interface to VoiceXML Media Services............................................................ 551 13.1 Introduction.................................................................................................. 551 13.2 VoiceXML Session Establishment and Termination................................... 553 13.3 Media Support..............................................................................................560 13.4 Returning Data to the Application Server................................................... 563 13.5 Outbound Calling.........................................................................................565 13.6 Call Transfer................................................................................................. 565 13.7 Security Considerations............................................................................... 569 13.8 IANA Considerations...................................................................................569 13.9 Summary......................................................................................................569 13.10 Problems....................................................................................................... 570 References................................................................................................................. 571 Chapter 14 Media Resource Control Protocol Version 2............................................................. 573 14.1 Introduction.................................................................................................. 573 14.2 Conventions.................................................................................................. 574 14.3 Architecture................................................................................................. 575 14.4 MRCPv2 Basics........................................................................................... 576 14.5 MRCPv2 Specification................................................................................. 583 14.5.5 Events........................................................................................... 587 14.6 MRCPv2 Generic Methods, Headers, and Result Structure........................ 588 14.7 Resource Discovery..................................................................................... 597 14.8 Speech Synthesizer Resource...................................................................... 597 14.10 Recorder Resource....................................................................................... 656 14.11 Speaker Verification and Identification........................................................663 14.12 Security Considerations...............................................................................684 14.13 IANA Considerations...................................................................................686 14.14 Examples......................................................................................................686 14.15 ABNF Normative Definition........................................................................697 14.16 XML Schemas.............................................................................................706 14.17 Summary...................................................................................................... 711 14.18 Problems....................................................................................................... 713 References................................................................................................................. 714 Chapter 15 Media Control Channel Framework (CFW) Call Flow Examples........................... 715 15.1 Introduction.................................................................................................. 715 15.2 Conventions.................................................................................................. 715 15.3 Terminology................................................................................................. 716 15.4 A Practical Approach................................................................................... 716 15.4.1 State Diagrams............................................................................. 716 15.5 Control Channel Establishment................................................................... 718 15.6 Use-Case Scenarios and Examples.............................................................. 726 15.7 Media Resource Brokering..........................................................................792 15.8 Security Considerations...............................................................................820 15.9 Summary......................................................................................................827 15.10 Problems.......................................................................................................830 Reference................................................................................................................... 831 Chapter 16 Multistream Immersive Telepresence Conferencing Systems.................................. 833 16.1 Requirements for Telepresence Multistreams.............................................. 833 16.2 Use Cases for Telepresence Multistreams.................................................... 837 16.3 Framework for Telepresence Multistreams.................................................. 845 16.4 XML Schema for the CLUE Data Model....................................................885 16.5 CLUE Protocol............................................................................................. 933 16.6 CLUE Protocol Data Channel.....................................................................966 16.7 Mapping RTP Streams to CLUE Media Captures.......................................980 16.8 Session Signaling for Controlling Multiple Streams for Telepresence (CLUE)......................................................................................................... 983 16.9 Summary......................................................................................................997 16.10 Problems..................................................................................................... 1001 References...............................................................................................................1002 Appendix....................................................................................................................................... 1005 Index............................................................................................................................................. 1013