Realizing the Metaverse

Wei Yang Bryan Lim, PhD, is a Wallenberg-NTU Presidential Postdoctoral Fellow, Nanyang Technological University, Singapore. He has received Best Paper Awards from the IEEE Wireless Communications and Networking Conference and the IEEE SPCC Technical Committee, and regularly serves as a reviewer for leading journals. Zehui Xiong, PhD, is an Assistant Professor at Singapore University of Technology and Design. He has published extensively, won many prestigious career/paper awards, and served as the editor for many leading journals in the areas of Internet of Things, edge computing and intelligence. He serves as the Associate Director of Future Communications R&D Programme. He was featured on the list of Forbes Asia 30 under 30 in 2023. Dusit Niyato, PhD, is a Professor in the School of Computer and Engineering, Nanyang Technological University, Singapore. He is an IEEE Fellow and serves as editor of numerous prestigious journals, including as editor-in-chief of IEEE Communications Surveys and Tutorials. Junshan Zhang, PhD, is a Professor in the Electrical and Computer Engineering Department at the University of California, Davis, USA. He has researched and published extensively on information networks, data science, 5g, wireless communications, and related subjects, and currently serves as editor-in-chief for IEEE Transactions on Wireless Communication. Xuemin (Sherman) Shen, PhD, is a University Professor in the Department of Electrical and Computer Engineering, University of Waterloo, Canada. He has published extensively on network resource management, wireless network security, Internet of Things, 5G, and more. He is a Fellow of the Engineering Institute of Canada, the Canadian Academy of Engineering, and the Royal Society of Canada, among others.

• About the Editors xiiiList of Contributors xviiForeword xixPreface xxi1 Introduction 1Wei Yang Bryan Lim, Zehui Xiong, Dusit Niyato, Junshan Zhang, and Xuemin Sherman Shen1.1 Introduction 11.1.1 Birth of the Metaverse 11.2 Architecture, Developments, and Tools of the Metaverse 41.2.1 Definition and Architecture 41.2.2 Current Developments 91.2.2.1 Massive Multiplayer Online Role-playing Games 101.2.2.2 Applications of the Metaverse 111.2.3 Tools, Platforms, and Frameworks 131.2.4 Summary and Lessons Learned 161.2.4.1 The Metaverse Cannot Be Realized by a Standalone Technology 161.2.4.2 Implementation Challenges 161.2.4.3 Standardized Protocols, Frameworks, and Tools 171.2.4.4 Metaverse Ecosystem and Economics 172 Communication and Computing in Edge-enabled Metaverse 19Minrui Xu, Wei Chong Ng, Wei Yang Bryan Lim, and Dusit Niyato2.1 Introduction 192.1.1 Rate–Reliability–Latency 3D Multimedia Networks 212.1.1.1 Resource Allocation for VR Streaming 252.1.1.2 Resource Allocation for AR Adaptation 282.1.1.3 Edge Caching for AR/VR Content 302.1.1.4 Hologram Streaming 312.1.2 Human-in-the-loop Communication 322.1.2.1 URLLC for the Tactile Internet 332.1.2.2 Semantic/Goal-aware Communication 342.1.3 Real-time Physical–Virtual Synchronization 362.1.3.1 Resource Allocation for Physical–Virtual Synchronization 372.1.3.2 Intelligent Edge Network-aided Virtual–Physical Synchronization 382.1.4 Lessons Learned 402.1.4.1 Efficient Immersive Streaming and Interaction 402.1.4.2 AI for Intelligent Edge Communication 402.1.4.3 Context-aware Immersive Content Delivery 402.1.4.4 Self-sustainable Physical–Virtual Synchronization 402.1.4.5 Decentralized Incentive Mechanism 412.2 Computation 412.2.1 The Cloud–Edge–End Computing Paradigm 432.2.2 Efficient AR/VR Cloud–Edge–End Rendering 452.2.2.1 Stochastic Demand and Network Condition 462.2.2.2 Stragglers at Edge Networks 482.2.2.3 Heterogeneous Tasks 492.2.3 Scalable AI Model Training 492.2.3.1 Parameter Pruning and Quantization 502.2.3.2 Low-rank Approximation 512.2.3.3 Knowledge Distillation 522.2.4 Privacy and Security 542.2.4.1 Trusted Execution Environment 552.2.4.2 Federated Learning 562.2.4.3 Adversarial Machine Learning 582.2.5 Lessons Learned 602.2.5.1 Adaptive AR/VR Cloud–Edge–End Rendering 602.2.5.2 On-demand and Generalized Model Compression 602.2.5.3 User-centric Computing 602.2.5.4 Secure Interoperable Computing 612.2.5.5 From Distribution to Decentralization 612.3 Summary 613 Advanced and Future Network Access Technologies for the Metaverse 63Andrea Garbugli, Domenico Scotece, Luca Foschini, and Paolo Bellavista3.1 Introduction 633.2 Edge-enabled Metaverse 643.2.1 Efficient Service Migration at the Edge 653.2.1.1 Multi-layer Container Migration 663.2.1.2 Metaverse-enabled Service Migration Architecture 683.2.1.3 Simulation Results 713.3 Ultra-low Latency Communications 733.3.1 ULL Networking Protocols 733.3.1.1 Time-sensitive Networking (TSN) 743.3.1.2 Wireless Networks: 5G and Wi-Fi 6 743.3.2 Acceleration Technologies 763.3.2.1 Linux eXpress Data Path (XDP) 763.3.2.2 Data Plane Development Kit (DPDK) 773.3.2.3 Remote Direct Memory Access (RDMA) 773.3.2.4 Integration for Metaverse Applications 783.3.3 SELENE: A Uniform Acceleration Middleware 783.3.3.1 SELENE: The APIs 793.3.3.2 SELENE: QoS Policies 813.3.3.3 SELENE: The Runtime 823.3.3.4 SELENE: Evaluation 853.3.3.5 Latency and Throughput Benchmarks 863.3.3.6 Lunar Streaming 903.4 Summary 924 How to Intelligentize the Metaverse 93Zhe Zhang, Jialin Wu, Ziwen Jin, Xidong Wang, Jiawen Kang, Yanchao Zhao, and Yi Wu4.1 Seven-layer Model of the Intelligentized Metaverse 934.1.1 Experience Layer 934.1.2 Exploration and Discovery Layer 944.1.3 Creator Economy Layer 954.1.4 Spatial Computing Layer 954.1.5 Decentralized Layer 954.1.6 Human–Computer Interaction Layer 964.1.7 Infrastructure Layer 964.2 AI for the Metaverse 964.2.1 Avatar Creation 964.2.2 Empowering 3D Chatbots 974.2.3 Language Processing 974.2.4 Enabling Human–Computer Interactions 974.2.5 3D Scene Rendering 984.3 Edge AI for the Metaverse 984.3.1 Edge AI Architectures 984.3.1.1 Cloud-edge Collaboration Architecture 984.3.1.2 Decentralized Architecture 994.3.1.3 Federated Learning Architecture 1004.3.2 How Does Edge AI Contribute to Creating the Metaverse? 1004.3.2.1 Unique Challenges in the Edge AI-empowered Metaverse 1004.3.2.2 Benefits of Edge AI for the Metaverse 1014.4 Edge AI-empowered Metaverse 1024.4.1 Cloud-edge-end Empowered Metaverse Network 1024.4.2 Mobile-edge-cloud Empowered Metaverse Network 1034.4.3 Decentralized Edge AI Empowered Metaverse Network 1044.4.4 Personalized and Heterogeneous Edge AI Empowered Metaverse Network 1054.4.5 Space–Air–Ground Empowered Metaverse Network 1064.5 AI and Edge AI Empowered Metaverse Applications 1074.5.1 Telecommuting 1074.5.2 Immersion Education 1084.5.3 Smart Cities 1084.5.4 Industrial Metaverse 1094.5.5 VR Tourism 1094.5.6 Digital Virtual Human 1094.6 Open Research Topics and Future Directions 1104.6.1 Ownership for AI-Generated Content 1104.6.2 Deepfakes and User Transparency 1104.6.3 The Right to be Forgotten 1104.6.4 Balance Between Virtual and Reality 1114.6.5 Security 1114.7 Summary 1125 How to Secure the Metaverse 113Zefeng Chen, Jiayi Sun, Jiayang Wu, Wensheng Gan, and Philip S. Yu5.1 Introduction 1135.2 Security Issues in the Metaverse 1175.2.1 Data Collection 1185.2.1.1 Solutions 1195.2.2 Data Transmission and Network Communication 1195.2.2.1 Solutions 1215.2.3 Data Processing 1225.2.3.1 Solutions 1225.2.4 Data Storage and Access 1235.2.4.1 Solutions 1235.3 Privacy Issues in the Metaverse 1245.3.1 Data Collection 1255.3.1.1 Solutions 1255.3.2 Data Usage and Access 1265.3.2.1 Solutions 1265.3.3 Data Transmission, Storage, and Sharing 1275.3.3.1 Solutions 1275.4 Open Problems and Opportunities 1285.4.1 Security Issues of 5G 1285.4.2 Security Issues of the Internet of Things 1295.4.3 Security Issues of Extended Reality Technology 1305.4.4 Security Issues of Cloud Computing 1315.5 Summary 132Bibliography 133Index 173