The research team of Assistant Professor James J.Q. Yu from the Department of Computer Science and Engineering at the Southern University of Science and Technology (SUSTech) has made a series of scientific progress in the field of privacy computing-empowered intelligent transportation systems. Two of the papers were published in IEEE Internet of Things Journal, a top international journal in this field.

Privacy-preserving transportation mode identification is one of the key challenges for intelligent transportation systems. With the rise of Federated Learning (FL), crowdsourcing-based FL has emerged as a cost-effective training solution to train powerful traffic pattern classifiers through a distributed approach without exposing the privacy of users’ data. However, existing transportation mode identification methods rely heavily on a large number of labeled traffic mode data, which is often difficult to achieve in the real world. Although numerous semi-supervised studies in recent years have attempted to address this problem by assigning Pseudo Labels to unlabeled data, this approach tends to degrade classification performance when large amounts of unlabeled data are involved.

In this regard, the study, entitled “Towards Crowdsourced Transportation Mode Identification: A Semi-Supervised Federated Learning Approach,” was published by Prof. Yu’s group in IEEE Internet of Things Journal.

The researchers proposed a semi-supervised FL scheme (MTSSFL) for transportation mode identification. The scheme consists of a novel semi-supervised FL framework and a deep neural network based on Ensemble Learning to achieve highly accurate and privacy-preserving crowdsourced transportation mode identification without relying on a large amount of labeled available data.

The algorithm introduces a consistency-updating algorithm in the training of FL, which inserts the global model into the gradient update of the local model with only unlabeled data to improve its training effect. In addition, a novel parameter aggregation mechanism (mean-teacher-averaging) was designed, which can further improve the traffic pattern recognition accuracy of the global model without additional training. Compared with traditional methods, this scheme can achieve higher accuracy with less training overhead while satisfying the privacy-preserving constraints of FL.

Figure 1. Ensemble learning-based transportation model identification model

Figure 2. Semi-supervised federated learning framework

Chenhan Zhang, a Research Assistant at SUSTech, is the first author of this paper, while Assistant Professor James J.Q. Yu is the corresponding author. Yuanshao Zhu, a master’s student at SUSTech, Christos Markos, a joint Ph.D. student at SUSTech and the University of Technology Sydney (UTS), and Professor Shui Yu at UTS are the co-authors. SUSTech is the first corresponding affiliation of the paper.

FL is widely adopted in traffic prediction tasks for large-scale IoT sensor data because its decentralized nature enables the privacy of data providers to be preserved. However, when deep learning-based traffic predictors are employed in FL systems, existing FL frameworks face excessive communication overhead in transmitting parameter updates of these models due to the large number of parameters contained in the deep learning models.

Prof. Yu’s team published a paper entitled “A Communication-Efficient Federated Learning Scheme for IoT-Based Traffic Forecasting,” which proposes a novel and practical FL Scheme (CTFed) to solve the above problem. The proposed scheme employs a Divide et Impera strategy to cluster customers based on the similarity between their local model parameters obtained through pre-training and parameter downscaling.

By integrating the Particle Swarm Optimization (PSO) algorithm, the scheme designs a two-step method to update the local models. It allows only one representative local model update in each cluster to be uploaded to the central server, thus significantly reducing the communication overhead of model update transmission in FL. In addition, the scheme is orthogonal to the mainstream gradient compression or sparsification-based approaches, and thus can further collaborate to optimize the communication overhead.

The experimental results show that the proposed scheme has excellent training efficiency, accurate prediction performance, and robustness to unstable network environments. It is expected to solve other research and industrial problems as a general distributed training scheme.

Figure 3. Communication efficient federated learning scheme

Chenhan Zhang is the first author of this paper, while James J.Q. Yu is the corresponding author. Researcher Lei Cui and Professor Yu Shui, both of UTS, are co-authors. SUSTech is the first corresponding affiliation of the paper.

Both studies were supported by the Shenzhen Natural Science Fund, Guangdong Basic and Applied Basic Research Foundation, and the Guangdong Provincial Key Laboratory of Braininspired Intelligent Computation.

Paper links (In order of appearance above):

IEEE Internet of Things Journal: https://ieeexplore.ieee.org/document/9632695

IEEE Internet of Things Journal: https://ieeexplore.ieee.org/document/9634121

To read all stories about SUSTech science, subscribe to the monthly SUSTech Newsletter.