A two-level fusion CNN model for classifying metro drivers’ distractions with functional near-infra-red spectroscopy and electrocardiography signals

This study develops a Convolutional Neural Network (CNN) -based two-level fusion model to identify cognitive distractions of metro drivers using their Electrocardiography (ECG) features and three types of functional near-infra-red spectroscopy (fNIRS) features (ΔOxyHb, ΔDeoxyHb, and ΔTotalHb). The model incorporates feature-level and decision-level fusions. Feature-level fusion combines ECG and fNIRS features to create a unified feature set, while decision-level fusion applies independent classifiers for ECG, fNIRS, and combined data to make final identification. For comparison, several alternative models are developed. Results indicate that the proposed two-level fusion model outperforms the non-fusion, feature-level fusion, and decision-level fusion models. Among the alternative models, those incorporating feature-level fusion outperform decision-level or non-fusion models. The feature-level fusion model that combines three types of fNIRS features demonstrates superior performance. Furthermore, all ECG features and 54.1% of fNIRS features show significant differences across the distraction levels. Drivers’ prefrontal cortex is more active during cognitive distractions.