Sleep accounts for one-third of a person’s life and is a key factor in maintaining long-term physiological and psychological health and quality of life. According to the 2024 White Paper on Sleep Health of Chinese Residents, nearly half of the Chinese population experiences sleep-related issues, many of which go unnoticed, placing them at risk of cardiovascular disease and mental illness. Therefore, sleep monitoring has emerged as a crucial method to improve sleep health, including sleep staging and identifying sleep-related breathing disorders.
Associate Professor Wenjin Wang from the Department of Biomedical Engineering at the Southern University of Science and Technology (SUSTech), in collaboration with Professor Fu Xiao from Nanjing University of Posts and Telecommunications (NJUPT), has made advances in the field of contactless health monitoring research.
Their work, titled “Camera-based Bi-modal PPG-SCG: Sleep Privacy-protected Contactless Vital Signs Monitoring”, has been published in the prestigious SCI journal IEEE Internet of Things Journal (JCR Q1).
Figure 1. The schematic diagram of privacy-protected contactless video sleep monitoring system
The polysomnography (PSG) system, which relies on wearable sensors, is the current clinical standard for diagnosing sleep disorders. However, it faces three major challenges in its application. Firstly, PSG attaches biosensors to the skin, which disrupts sleep quality and fails to accurately reflect the individual’s natural sleep quality. Secondly, wearing PSG is cumbersome and complex, often taking over thirty minutes and requiring operation from clinicians, making large-scale adoption challenging. Lastly, PSG diagnosis relies on manual interpretation by clinicians, leading to subjective bias, and is both time-consuming and labor-intensive.
This has driven research on contactless monitoring methods of physiological parameters using optical cameras, millimeter-wave radar, and Wi-Fi sensors to explore their feasibility in sleep monitoring. While cameras have advantages in offering accuracy and robustness among these sensors, privacy concerns during sleep pose a challenge for their practical application.
Figure 2. Bi-modal and uni-modal principles in physiological monitoring
The researchers creatively proposed a novel approach to contactless sleep monitoring with built-in privacy protection using a dual-defocused camera system. This method captures facial PPG signals using a defocused camera (CamPPG), which protects sleep privacy while detecting signals originated from the distal end of the heart. Another defocused camera was used to capture SCG signals (CamSCG), which emanated from the proximal ends of the heart. This enabled a bi-modal monitoring setup (PPG-SCG) that provided a comprehensive view of cardiovascular parameters.
Through this dual-camera configuration, pulse transit time (PTT) was calculated by determining the phase difference between the CamSCG and CamPPG signals, allowing for the prediction of blood pressure values, including systolic and diastolic pressure. Additionally, a mmWave radar was employed to capture the proximal SCG (RFSCG), serving as a benchmark against CamSCG. This led to two bi-modal setups—CamSCG-CamPPG and RFSCG-CamPPG—providing the measurement of multiple parameters including respiratory rate (RR), heart rate (HR), heart rate variability (HRV), and PTT.
Regression models were established based on interpretable machine learning algorithms that associate multiple features, especially PTT, with blood pressure (BP), enabling calibrated BP estimation. The random forest regression model based on the CamSCG-CamPPG bi-modality achieved the most accurate personalized BP estimation. The mean absolute error (MAE) ± standard deviation (STD) was 3.99±5.26 mmHg for systolic blood pressure (SBP) and 3.70±4.79 mmHg for diastolic blood pressure (DBP). For HR estimation using CamSCG, the MAE±STD was 1.3±1.6 bpm, and the R-value was 0.99, while RR estimation showed an MAE±STD of 4.2±3.5 bpm, with an R-value of 0.83.
This research successfully proposed and verified the fusion of camera-based PPG and SCG signals for multi-variate contactless monitoring of multiple sleep-related parameters (including RR, HR, HRV and BP), while also preserving sleep privacy protection through the use of defocused cameras. This approach provides new insights for advancements in contactless video sleep monitoring products, targeting seamless, intelligent, and comfortable sleep staging and the detection of sleep-related breathing disorders.
Figure 3. Remote ICU monitoring technology in several elderly care communities
Graduate student Yingen Zhu from the Department of Biomedical Engineering at SUSTech is the first author of the paper. Associate Professor Wenjin Wang and Professor Fu Xiao are the co-corresponding authors, with SUSTech serving as the first affiliation institution of the paper.
Paper link: https://ieeexplore.ieee.org/document/10729858
To read all stories about SUSTech science, subscribe to the monthly SUSTech Newsletter.
