Real-time eye state recognition using dual convolutional neural network ensemble

xlomafota.13 shared this article with you from Inoreader Real-time eye state recognition using dual convolutional neural network ensemble Via Latest Results for Journal of Real-Time Image Processing Abstract Automatic recognition of the eye states is essential for diverse computer vision applications related to drowsiness detection, facial emotion recognition (FER), human–computer interaction (HCI), etc. Existing solutions for eye state detection are either parameter intensive or suffer from a low recognition rate. This paper presents the design and implementation of a vision-based system for real-time eye state recognition on a resource-constrained embedded platform to tackle these issues. The designed system uses an ensemble of two lightweight convolutional neural networks (CNN), each trained to extract relevant information from the eye patches. We adopted transfer-learning-based fine-tuning to overcome the over-fitting issues when training the CNNs on small sample eye state datasets. Once trained, these CNNs are integrated and jointly fine-tuned to achieve enhanced performance. Experimental results manifest the effectiveness of the proposed eye state recognizer that is robust and computationally efficient. On the ZJU dataset, the proposed DCNNE model delivered the state-of-the-art recognition accuracy of 97.99% and surpassed the prior best recognition accuracy of 97.20% by 0.79%. The designed model also achieved competitive results on the CEW and MRL datasets. Finally, the designed CNNs are optimized and ported on two different embedded platforms for real-world applications with real-time performance. The complete system runs at 62 frames per second (FPS) on an Nvidia Xavier device and 11 FPS on a low-cost Intel NCS2 embedded platform using a frame size of 640 \(\times\) 480 pixels resolution. View on the web Inoreader. Take back control of your news feed. Follow us on Twitter and Facebook.