Towards Drowsiness Driving Detection Based on Multi-Feature Fusion and LSTM Networks

Drowsiness driving poses a huge threat to the traffic safety. In this paper, a novel drowsiness driving detection method based on multi-feature fusion and long short-term memory (LSTM) recurrent neural networks is proposed to reduce traffic accidents caused by drowsiness driving. Firstly, we collect steering wheel angles (SWAs) of vehicles and facial videos of drivers by a driving simulator. Secondly, the drowsiness driving-related steering features and facial expression features are respectively extracted from the collected SWAs and facial videos by the One Way ANOVA method and the FEFENet network, and then they are fused by concatenation operation. Considering that the generation of drowsiness is a long-term dynamic process and the degree of drowsiness accumulates over time, we design LSTM networks to cope with the fused feature sequence in a fixed duration, thereby establishing a effective drowsiness driving detection model. Some experiments are conducted to validate the performance of the proposed method, and the results demonstrate that our method can get robust and high accuracy performance in many challenging driving scenarios.