Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immersive Telepresence using SIP

Radhika Ranjan Roy has been an electronics engineer, United States Army Research, United StatesCommand, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance,and Reconnaissance (C5ISR)-S&TCD Laboratories (previously known as CERDEC), AberdeenProving Ground (APG), Maryland, United States since 2009. Dr. Roy is leading research and developmentefforts in the development of scalable large-scale SIP-based VoIP/Multimedia networks andservices, cybersecurity, artificial intelligence and machine/deep learning (AI/ML/DL) architecture,mobile ad hoc networks (MANETs), Peer-to-Peer (P2P) networks, cybersecurity detecting applicationsoftware and network vulnerability, jamming detection, supporting an array of the Army/Department of Defense’s Nationwide and Worldwide Warfighter Networking Architectures andparticipating in technical standards development in Multimedia/Real-Time Services Collaboration,IPv6, Radio Communications, Enterprise Services Management, and Information Transfer ofDepartment of Defense (DoD) Technical Working Groups (TWGs). He received his PhD in ElectricalEngineering with Major in Computer Communications from the City University of New York(CUNY), NY, United States in 1984 and his MS in Electrical Engineering from the NortheasternUniversity, Boston, MA, United States in 1978. He received his BS in Electrical Engineering fromthe Bangladesh University of Engineering & Technology (BUET), Dhaka, Bangladesh in 1967. Hewas born in the renowned country town of Derai, Bangladesh.Prior to joining CERDEC, Dr. Roy worked as the lead system engineer at CACI, Eatontown,NJ from 2007 to 2009 and developed Army Technical Resource Model (TRM), Army EnterpriseArchitecture (AEA), DoD Architecture Framework (DoDAF), and Army LandWarNet (LWN)Capability Sets, and technical standards for Joint Tactical Radio System (JTRS), Mobile IPv6,MANET, and SIP, supporting Army Chief Information Officer (CIO)/G-6. Dr. Roy worked as seniorsystem engineer, SAIC, Abingdon, MD from 2004 to 2007, supporting Modeling, Simulations,Architectures, and System Engineering of many Army projects: WIN-T, FCS, and JNN.During his career, Dr. Roy worked in AT&T/Bell Laboratories, Middletown, NJ as senior consultantfrom 1990 to 2004 and led a team of engineers in designing AT&T’s Worldwide SIP-basedVoIP/Multimedia Communications Network Architecture, consisting of wired and wireless, fromthe preparation of Request for Information (RFI) to the evaluation of vendor RFI responses andinteractions with all selected major vendors related to their products. He participated in and contributedto the development of VoIP/H.323/SIP multimedia standards in ITU-T, IETF, ATM, andFrame Relay standard organizations.Dr. Roy worked as senior principal engineer in CSC, Falls Church, VA from 1984 to 1990 andworked in the design and performance analysis of the US Treasury nationwide X.25 packet-switchingnetwork. In addition, he designed many network architectures of many proposed U.S. Governmentand Commercial Worldwide and Nationwide Networks: Department of State TelecommunicationsNetwork (DOSTN), U.S. Secret Service Satellite Network, Veteran Communications Network, andFord Company’s Dealership Network. Prior to CSC, he worked from 1967 to 1977 as deputy director,Design, in PDP, Dhaka, Bangladesh.Dr. Roy’s research interests include the areas of artificial intelligence, machine/deep learning,blockchain cloud and fog computing, mobile ad hoc networks, multimedia communications, peer-topeernetworking, and quality-of-service. He has published over 60 technical papers and is holding orpending over 35 patents. He also participates in many IETF working groups. Dr. Roy authored threebooks Handbook on SDP for Multimedia Session Negotiations: SIP and WebRTC Telephony (CRCPress/Taylor & Francis, 2018), Handbook on Session Initiation Protocol: Networked MultimediaCommunications for IP Telephony (CRC Press/Taylor & Francis, 2016), and Handbook of MobileAd Hoc Networks for Mobility Models, Springer, 2010. He lives in the historical district of HowellTownship, New Jersy, with his wife Jharna.

• ContentsPreface...........................................................................................................................................xxxiAuthor...........................................................................................................................................xxxvChapter 1 Scalable Networked Multipoint Multimedia Conferencing and Telepresence..............11.1 Networked Multipoint Multimedia Conferencing..............................................11.2 Basic Conferencing Protocol Architecture.........................................................41.3 Third-Party Multiparty Conferencing.............................................................. 121.4 Multistream Immersive Telepresence...............................................................231.5 Scalable Centralized Multipoint Multimedia Conferencing Systemsand Protocols Framework.................................................................................241.6 Terminology.....................................................................................................281.7 Summary..........................................................................................................281.8 Problems........................................................................................................... 41References................................................................................................................... 42Chapter 2 Centralized Conferencing System Architecture......................................................... 432.1 Introduction...................................................................................................... 432.2 Convention........................................................................................................ 432.3 Terminology..................................................................................................... 432.4 Overview..........................................................................................................442.5 Centralized Conferencing Data........................................................................ 452.6 Centralized Conferencing Constructs and Identifiers...................................... 472.7 Conferencing System Realization....................................................................502.8 Conferencing Mechanisms............................................................................... 552.9 Conference Scenario Realizations....................................................................562.10 Relationships between SIP and Centralized Conferencing SystemFrameworks......................................................................................................692.11 Security Considerations....................................................................................692.12 Summary.......................................................................................................... 712.13 Problems........................................................................................................... 72Chapter 3 Media Server Control Architecture............................................................................. 733.1 Introduction...................................................................................................... 733.2 Terminology..................................................................................................... 753.3 Architecture Overview..................................................................................... 753.4 NSIP Usage.......................................................................................................773.5 Media Control for IVR Services...................................................................... 783.6 Media Control for Conferencing Services........................................................803.7 Security Considerations....................................................................................853.8 Summary..........................................................................................................863.9 Problems...........................................................................................................87Chapter 4 Conferencing Information Data Model for Centralized Conferencing.......................894.1 Introduction......................................................................................................894.2 Terminology.....................................................................................................894.3 Overview..........................................................................................................89Comparison.......................................................................... 914.4 Data Model Definition......................................................................................924.5 RELAX NG Schema...................................................................................... 1034.6 XML Schema Extensibility............................................................................ 1034.7 XML Example................................................................................................ 1034.8 Non-Normative RELAX NG Schema in XML Syntax (Appendix A ofRFC 6501)....................................................................................................... 1034.9 Relax Schema in XML Syntax (Appendix B of RFC 6501).......................... 1034.10 Security Considerations.................................................................................. 1044.11 Summary........................................................................................................ 1084.12 Problems......................................................................................................... 109References................................................................................................................. 111Chapter 5 Centralized Conferencing Manipulation Protocol.................................................... 1295.1 Introduction.................................................................................................... 1295.2 Conventions and Terminology........................................................................ 1295.3 XCON Conference Control System Architecture.......................................... 1295.4 Protocol Overview.......................................................................................... 1315.5 CCMP Messages............................................................................................ 1345.6 Example of CCMP in Action......................................................................... 1535.7 Locating a Conference Server........................................................................ 1625.8 Managing Notifications.................................................................................. 1635.9 HTTP Transport............................................................................................. 1655.10 Security Considerations.................................................................................. 1665.11 XML Schema................................................................................................. 1685.12 IANA Considerations..................................................................................... 1695.13 Summary........................................................................................................ 1765.14 Evaluation of Other Protocol Models and Transports Considered forCCMP (Appendix A of RFC 6503)................................................................ 1765.15 Problems......................................................................................................... 178References................................................................................................................. 179Chapter 6 Binary Floor Control Protocol.................................................................................. 1816.1 Introduction.................................................................................................... 1816.2 Terminology................................................................................................... 1816.3 Scope.............................................................................................................. 1816.4 Overview of Operation................................................................................... 1836.5 Packet Format................................................................................................. 1866.6 Transport........................................................................................................2006.7 Lower-Layer Security..................................................................................... 2016.8 Protocol Transactions..................................................................................... 2016.9 Authentication and Authorization..................................................................2026.10 Floor Participant Operations..........................................................................2026.11 Chair Operations............................................................................................2056.12.1 Requesting Information about Floors...............................................2066.13 Floor Control Server Operations....................................................................2086.14 Security Considerations.................................................................................. 2146.15 BFCP Connection Establishment................................................................... 2156.16 IANA Considerations..................................................................................... 2186.17 Summary........................................................................................................ 2216.18 Problems.........................................................................................................222Chapter 7 XCON Notification Service.......................................................................................2257.1 Introduction....................................................................................................2257.2 Terminology...................................................................................................2257.3 Notification Formats.......................................................................................2257.4 Full Notifications............................................................................................2257.5 Partial Notifications........................................................................................2277.6 IANA Considerations.....................................................................................2297.7 Security Considerations..................................................................................2297.7.1 Connection Security..........................................................................2307.7.2 Authorization Considerations ...........................................................2307.8 Summary........................................................................................................2307.9 Problems......................................................................................................... 231Reference................................................................................................................... 231Chapter 8 Media Channel Control Framework.......................................................................... 2338.1 Introduction.................................................................................................... 2338.2 CFW Capabilities........................................................................................... 2338.3 Overview........................................................................................................ 2358.4 Control Channel Setup................................................................................... 2388.4.1 Control Client SIP UAC Behavior..................................................... 2388.4.2 Control Server SIP UAS Behavior.................................................... 2398.5 Establishing Media Streams – Control Client SIP UAC Behavior.................2408.6 Control Framework Interactions..................................................................... 2418.7 Response Code Descriptions.......................................................................... 2478.8 Control Packages............................................................................................ 2478.9 Formal Syntax................................................................................................2498.10 Examples........................................................................................................ 2518.11 Extensibility....................................................................................................2548.12 Security Considerations..................................................................................2548.12.1 Session Establishment.......................................................................2548.13 IANA Considerations.....................................................................................2568.14 Common Package Components (Appendix A of RFC 6230).........................2568.15 Summary........................................................................................................ 2578.16 Problems......................................................................................................... 257Chapter 9 Mixer Package for Media Control Channel Framework........................................... 2599.1 Introduction.................................................................................................... 2599.2 Conventions and Terminology........................................................................2609.3 Control Package Definition............................................................................2609.4 Element Definitions........................................................................................ 2629.5 Formal Syntax................................................................................................2909.6 Examples........................................................................................................ 2919.6.1 AS-MS Framework Interaction Examples........................................ 2919.6.1.1 Creating a Conference Mixer and Joining aParticipant..........................................................................3099.6.1.2 Receiving Active Talker Notifications...............................3099.6.1.3 Conference Termination....................................................3099.6.2 Mixing Examples.............................................................................. 3109.6.2.1 Audio Conferencing........................................................... 3109.6.2.2 Bridging Connections........................................................ 312xvi Contents9.6.2.3 Video Conferencing........................................................... 3129.7 Security Considerations.................................................................................. 3139.8 IANA Considerations..................................................................................... 3159.9 Summary........................................................................................................ 3159.10 Problems......................................................................................................... 315References................................................................................................................. 316Chapter 10 Media Session Recording.......................................................................................... 31710.1 Use Cases and Requirements for SIP-Based Media Recording................... 31710.2 Media Recording Architecture....................................................................32410.3 Media Recording Metadata.......................................................................... 331XML Body.................................................................34410.3.9 XML Schema Definition for Recording Metadata....................... 34510.3.10 Security Considerations................................................................34810.3.11 IANA Considerations...................................................................34910.4 Media Session Recoding Protocol...............................................................34910.5 Summary...................................................................................................... 37110.6 Problems....................................................................................................... 372References................................................................................................................. 373Chapter 11 Media Resource Brokering....................................................................................... 37511.1 Introduction.................................................................................................. 37511.2 Conventions and Terminology..................................................................... 37511.3 Problem Discussion...................................................................................... 37711.4 Deployment Scenario Options..................................................................... 37811.5 MRB Interface Definitions.......................................................................... 38111.6 MRB Acting as a B2BUA............................................................................40511.7 Multimodal MRB Implementations.............................................................40511.8 Relative Merits of Query Mode, IAMM, and IUMM.................................40611.9 Examples......................................................................................................40711.10 Media Service Resource Publisher Interface XML Schema....................... 42511.11 Media Service Resource Consumer Interface XML Schema......................44411.12 Security Considerations............................................................................... 46111.13 IANA Considerations...................................................................................46311.14 Summary......................................................................................................46311.15 Problems.......................................................................................................464References.................................................................................................................465Chapter 12 Media Control Channel Framework for Interactive Voice Response ....................... 46712.1 Introduction.................................................................................................. 46712.2 Conventions and Terminology.....................................................................46912.3 Control Package Definition..........................................................................46912.4 Element Definitions...................................................................................... 47112.5 Formal Syntax.............................................................................................. 51112.6 Examples...................................................................................................... 53412.8 IANA Considerations...................................................................................54412.9 Using VoiceXML as a Dialog Language.....................................................54412.10 Summary...................................................................................................... 54912.11 Problems....................................................................................................... 550Chapter 13 SIP Interface to VoiceXML Media Services............................................................ 55113.1 Introduction.................................................................................................. 55113.2 VoiceXML Session Establishment and Termination................................... 55313.3 Media Support..............................................................................................56013.4 Returning Data to the Application Server................................................... 56313.5 Outbound Calling.........................................................................................56513.6 Call Transfer................................................................................................. 56513.7 Security Considerations............................................................................... 56913.8 IANA Considerations...................................................................................56913.9 Summary......................................................................................................56913.10 Problems....................................................................................................... 570References................................................................................................................. 571Chapter 14 Media Resource Control Protocol Version 2............................................................. 57314.1 Introduction.................................................................................................. 57314.2 Conventions.................................................................................................. 57414.3 Architecture................................................................................................. 57514.4 MRCPv2 Basics........................................................................................... 57614.5 MRCPv2 Specification................................................................................. 58314.5.5 Events........................................................................................... 58714.6 MRCPv2 Generic Methods, Headers, and Result Structure........................ 58814.7 Resource Discovery..................................................................................... 59714.8 Speech Synthesizer Resource...................................................................... 59714.10 Recorder Resource....................................................................................... 65614.11 Speaker Verification and Identification........................................................66314.12 Security Considerations...............................................................................68414.13 IANA Considerations...................................................................................68614.14 Examples......................................................................................................68614.15 ABNF Normative Definition........................................................................69714.16 XML Schemas.............................................................................................70614.17 Summary...................................................................................................... 71114.18 Problems....................................................................................................... 713References................................................................................................................. 714Chapter 15 Media Control Channel Framework (CFW) Call Flow Examples........................... 71515.1 Introduction.................................................................................................. 71515.2 Conventions.................................................................................................. 71515.3 Terminology................................................................................................. 71615.4 A Practical Approach................................................................................... 71615.4.1 State Diagrams............................................................................. 71615.5 Control Channel Establishment................................................................... 71815.6 Use-Case Scenarios and Examples.............................................................. 72615.7 Media Resource Brokering..........................................................................79215.8 Security Considerations...............................................................................82015.9 Summary......................................................................................................82715.10 Problems.......................................................................................................830Reference................................................................................................................... 831Chapter 16 Multistream Immersive Telepresence Conferencing Systems.................................. 83316.1 Requirements for Telepresence Multistreams.............................................. 83316.2 Use Cases for Telepresence Multistreams.................................................... 83716.3 Framework for Telepresence Multistreams.................................................. 84516.4 XML Schema for the CLUE Data Model....................................................88516.5 CLUE Protocol............................................................................................. 93316.6 CLUE Protocol Data Channel.....................................................................96616.7 Mapping RTP Streams to CLUE Media Captures.......................................98016.8 Session Signaling for Controlling Multiple Streams for Telepresence(CLUE)......................................................................................................... 98316.9 Summary......................................................................................................99716.10 Problems..................................................................................................... 1001References...............................................................................................................1002Appendix....................................................................................................................................... 1005Index............................................................................................................................................. 1013