Research

Our question

We ask why a model fails when the subject, the recording device or the recording session changes, and how to prevent that failure. Real observations carry three defects: missing data, misaligned channels and distribution shift. Instead of patching them with hand-tuned preprocessing that only fits the data it was tuned on, we build their handling into the model itself, so that the space in which the model represents its input — the latent space — does not change with acquisition conditions.

Missing data
Acquisition cannot be fully controlled: sensors break, are occluded or disconnect.

Channel misalignment
The system is dynamic, so the regions seen by different modalities do not line up.

Distribution shift
Subjects and sessions differ in their statistics.

Current direction: robust EEG feature extraction under multimodal constraints — using microscopic neural recordings (calcium imaging, ECoG) during training to constrain the EEG latent space, so that brain–computer interfaces stay reliable across people and sessions while needing only EEG at inference.

Four steps along one chain: why latent spaces fail, how to make them robust within one dataset, across data sources and time, and across people.

1 · Why the latent space is not robust

A machine-and-human reader study (Nature Communications, 2021, first author) used a generative adversarial network to change only the lesion region of mammograms. Synthetic images turned 69.1% of correctly diagnosed cases into wrong diagnoses, and five breast radiologists recognised 29%–71% of them. Some synthetic lesions sat where no breast tissue exists, yet the models scored them like real ones: their latent space does not encode anatomical structure. The work was first presented orally at RSNA 2019 and led to an NSFC General Program grant on judging the plausibility of synthetic medical images.

Try the online demo →

Real mammograms (top), GAN-synthesised versions with a lesion inserted or removed (middle, arrows) and their difference (bottom). Zhou Q, et al. Nat Commun 2021, 12: 7281, Fig. 2 (CC BY 4.0).

2 · A robust latent space within one dataset

In a feed-forward network the designer controls three things: the architecture, the activation function and the loss. We let each layer choose its own activation function from the network structure and the task (IEEE TNNLS, 2023, corresponding author), separate redundant from missing feature components with RMFDNet (Engineering Applications of AI, 2025, first author), and give correct gradients on soft edges with a non-binary IoU loss (Expert Systems with Applications, 2023, first author).

RMFDNet predicts the missing and the redundant features of a segmentation separately and repairs them. Zhou Q, et al. Eng Appl Artif Intell 2025, 139: 109459, Fig. 1.

3 · Stable across hospitals and time points

A multicentre study (eClinicalMedicine, 2025, co-corresponding author; master's student Chen Jin co-first author) predicts the response of breast cancer to neoadjuvant chemotherapy from MRI before and early in treatment. A Siamese network extracts features at both time points and a Transformer models their interaction. Over 1,044 patients from 5 centres, the AUC on three external validation cohorts was 0.892–0.923, and in two independent public cohorts the predicted responders showed up-regulated immune genes.

Try the online demo →

Spatiotemporal interaction model (top) and internal/external validation (bottom). Tang W, Jin C, et al. eClinicalMedicine 2025, 85: 103298, part of Fig. 2 (CC BY 4.0).

4 · Stable across people

EEG differs greatly between people, which keeps brain–computer interfaces from practical use. Ph.D. student Kunkun Zhang proposed a masked self-supervised contrastive framework for motor-imagery EEG (ICASSP 2025), whose three parts map onto the three data defects, and an online test-time adaptation method that updates the decoder on each new EEG segment in 40–60 ms, improving accuracy by 2.22% and 3.67% on two public datasets (IEEE Transactions on Instrumentation and Measurement, 2026).

Online adaptation framework: offline training (top), online testing (bottom). Zhang K, Wu K, Chen Z, Hu H, Zhou Q. IEEE TIM 2026, 75: 4015112, accepted manuscript Fig. 2.

Now · Cross-scale constraints for EEG

From 2024 to 2026 the PI worked at Weill Cornell Medicine on simultaneous widefield calcium imaging, two-photon imaging and transparent ECoG in mouse cortex. A co-first-author preprint (bioRxiv, 2025) showed that suppressing cortical excitatory neurons produces large synchronous calcium transients together with focal ECoG spikes. Because microscopic population activity and macroscopic field potentials correspond, microscopic recordings can constrain the EEG latent space during training; at inference only EEG is needed.

Experimental pipeline and simultaneous widefield calcium imaging with ECoG. Wu S#, Zhou Q#, et al. bioRxiv 2025, doi:10.1101/2025.10.02.676593, Fig. 1 (# co-first authors, CC BY-NC 4.0).

Earlier lines of work

Vibration-to-sound seismic sensor

Ground vibration moves a proof mass (the device battery) onto a membrane in a sealed chamber; a MEMS microphone records the sound. Half the weight of a moving-coil geophone, 58 times its sensitivity at 70 Hz, ≤ 726 µW (IEEE Trans. Ind. Electron., 2017).

Ultra-dense microphone arrays

A 10 × 10 array with 5 mm spacing finds the direction of low-frequency sources whose wavelength (> 0.686 m) is more than ten times the array width (< 0.064 m); mean error 2.797° in noisy simulation (Meas. Sci. Technol., 2025, Ph.D. student Kunkun Zhang).

Wind-turbine blades from a latent space

Blade cross-sections are encoded as images and optimised by a genetic algorithm with CFD. The search grew two separate islands beside the main blade that add thrust; efficiency +13.77% and +21.11% at two tip-speed ratios (Soft Computing, 2018; Neurocomputing, 2022).

Multimodal biometric terminal

A hand-held terminal that recognises fingerprint, face, iris, gait and voice, built with the Shanghai Institute of Microsystem and Information Technology (CAS); cloud service software since 2025.