RIS CUTTING EDGE FOURM

LLM/Foundation Model Empowered Synesthesia of Machines (SoM): AI-native Intelligent Multi-Modal Sensing-Communication Integration

LLM/基座模型赋能机器联觉:AI原生的通信与多模态感知智能融合

LLM/Foundation Model Empowered Synesthesia of Machines (SoM): AI-native Intelligent Multi-Modal Sensing-Communication Integration

To support future intelligent multifunctional sixth-generation (6G) wireless communication networks,Synesthesia of Machines (SoM) is proposed as a novel paradigm for artificial intelligent(AI)-native intelligent multi-modal sensing-communication integration. However, existing SoM system designs rely on task-specific Al models and face challenges such as scarcity of massive high-quality datasets,constrained modeling capability, poor generalization, and limited universality. Recently,foundation models (FMs) have emerged as a new deep learning paradigm and have been preliminarily applied to SoM-related tasks, but a systematic design framework is still lacking. First, the report will introduce the basic concepts of SoM and systematically introduce two roadmaps for empowering SoM with LLM/foundation models, including large language models (LLMs), and SoM domain-specific FMs( Wireless Foundation Models). Second, the report will introduce the latest large-scale, high-fidelity intelligent multi-modal sensing-communication integration dataset SynthSoM, and focus on FM-empowered SoM transceivers based on this dataset, including LLM-based and wireless foundation model-based design.For LLM-based design, the report will propose LLM for path loss generation (LLM4PG), LLM for wireless multi-task SoM transceiver (LLM4WM).For wireless foundation model-based design, the report will introduce wireless foundation model (WiFo) for channel prediction. Additionally, the report will present the first hardware demo of a video transmission system embedded with a WiFo.

Cheng Xiang, Peking University Boya Distinguished Professor (Long term Professor), IEEE Fellow,AAIA Fellow, Fellow of the Chinese Association of Automation, Fellow of the China Institute of Communications. He has received National Science Fund for Distinguished Young Scholars, Xplorer Prize 2023 Award, China Frontiers of Engineering Young Scholar by the Chinese Academy of Engineering, IEEE Asia-Pacific Outstanding Young Researcher Award , Top 0.05% Global Scientists by ScholarGPS and Highly Cited Chinese Researchers by Elsevier. His main research directions focus on the deep cross integration of communication networks and artificial intelligence, including intelligent communication networks and connected intelligence. Currently, he serves as an executive director of the Chinese Association of Automation and the chairman of the CAA Committee on Connected Intelligence. He has published over 300 papers, including more than 100 in IEEE journals, such as 4 ESI hot papers and 21 ESI highly cited papers. He has also published 9 English monographs and 3 Chinese monographs. Google Scholar has been cited 16180 times, h-index=65,i10-index=231. As the first author, he won the first prize of Natural Science of the China Institute of Communications and the first prize of Natural Science from the Chinese Association of Automation.