Deep Learning Enabled Semantic Communications

Zhijin Qin is an Associate Professor with Tsinghua University, China. She is an Associate Editor for IEEE Transactions on Communications, IEEE Transactions on Cognitive Networking, and IEEE Communications Letters. Huiqiang Xie, PhD, is an Associate Professor at Jinan University, Guangzhou, Guangdong, China. Zhenzi Weng is a Postdoctoral researcher at Imperial College London, UK. Xiaoming Tao is a Professor with the Department of Electronic Engineering at Tsinghua University. She is also a Senior Member of the IEEE.

• Foreword ixPreface xiAcknowledgments xvAcronyms xviiNotation xxi1 Introduction 11.1 Conventional Communications versus Semantic Communications 21.1.1 Three-level Communications 21.1.2 History of Semantic Communications 31.2 Introducing Deep Learning to Semantic Communications 41.2.1 Deep Learning Basics 41.2.2 Deep Learning Enabled Semantic Communications 111.3 Semantic Communications for Further Networks 13References 152 Semantic Information Theory 192.1 Semantic Entropy 192.1.1 Logical Probability Based 192.1.2 Synonymous Mapping Based 212.1.3 Fuzzy Theory Based 232.1.4 Task Based 242.2 Semantic Channel Capacity 242.2.1 Logical Probability Based 242.2.2 Synonymous Mapping Based 252.3 Semantic Source Coding Theorem 262.3.1 Logical Probability Based 262.3.2 Synonymous Mapping Based 272.4 Semantic Channel Coding Theorem 282.4.1 Logical Probability Based 282.4.2 Synonymous Mapping Based 282.5 Information Bottleneck 292.5.1 Classical Information Bottleneck 292.5.2 Knowledge Collision-based Information Bottleneck 30References 303 Joint Semantic-channel Coding for Source Reconstruction 333.1 Semantic Communications for Text 343.1.1 Joint Semantic-channel Coding for Text 353.2 Semantic Communications for Speech 383.2.1 Joint Semantic-channel Coding for Speech 393.3 Semantic Communications for Image 423.3.1 Joint Semantic-channel Coding for Image 423.4 Performance Metrics 483.4.1 Performance Metrics for Text Accuracy 483.4.2 Performance Metrics for Speech Quality 493.4.3 Performance Metrics for Image Quality 49References 524 Task-oriented Semantic Communications 554.1 Single-modal Task-oriented Semantic Communications 554.1.1 Semantic Communications for Machine Translation 564.1.2 Semantic Communications for Speech Recognition and Synthesis 594.2 Multimodal Task-oriented Semantic Communications 694.2.1 Semantic Communication Systems for Visual Question Answering 69References 745 Joint Sensing and Semantic Communications 775.1 Introduction and Framework of Joint Sampling and Coding 775.1.1 Semantic Sampling 785.1.2 Semantic Reconstruction 795.2 Joint Semantic Sampling and Coding for Image 795.2.1 Semantic-aware Image Compressed Sensing 805.2.2 Adaptive Sampling and Semantic-channel Coding 845.3 Joint Semantic Sampling and Coding for Video 895.3.1 Semantic-based Video Sampling and Reconstruction 92References 956 Semantic Impairments in Communications 976.1 JSCC Framework with Semantic Impairments 986.2 Source Semantic Impairments Suppression 1006.2.1 Robust Semantic Communications for Text 1006.2.2 Robust Semantic Communications for Speech 1066.2.3 Robust Semantic Communications for Image 1136.3 Knowledge Base Semantic Impairments Suppression 1206.3.1 Robust SKB 120References 1257 Generative AI-enabled Semantic Communications 1297.1 Introducing Generative Models to Semantic Communications 1297.2 Framework of Generative Semantic Communications 1317.2.1 Main Components of Generative Semantic Communication System 1337.2.2 Key Interactions and Processes 1357.3 Demonstration of Generative Semantic Communication for Video Conferencing 1367.4 Applications of Semantic Communications in Other Scenarios 1387.4.1 Immersive Communications 1387.4.2 Autonomous Driving 1397.4.3 Smart Cities 1407.4.4 Satellite Networks 141References 1428 Conclusion and Challenges 145Index 149