Sleepiness and Emotion Detection with CNN and MediaPipe
Session Number
Project ID: CMPS 10
Advisor(s)
Victor Elarde; Northrop Grumman
Tao Zhang; Northrop Grumman
Discipline
Computer Science
Start Date
20-4-2022 10:05 AM
End Date
20-4-2022 10:20 AM
Abstract
According to research by Steinhauser et. al, inattention of the driver due to additional tasks, emotion changes, fatigue or eye movements played an important role in 78% of car accidents on the road. The National Highway Traffic Safety Administration estimates that at least 100,000 police-reported crashes are the direct result of driver fatigue each year. This equates to approximately 1,550 deaths, 71,000 injuries, and $12.5 billion in monetary losses. The goal of this SIR is to develop a system that detects and alerts unstable emotions and drowsiness during daily driving, in order to reduce that fatality on the road, saving dozens of lives. The SIR project monitors and analyzes driver’s facial features using real time video and alert drive when drowsiness or unstable mood is detected. The project, based on open-source code, has integrated libraries of face-meshing (MediaPipe), emotion detection, TensorFlow (CNN), and the eye-aspect-ratio algorithm we developed. It classifies multiple emotions and identifies drowsiness effectively. Future explorations can be in the area of automatic driver’s mode control. Emotion detection can serve as a regulator to adjust automotive informatics and climate systems to help stabilize drivers’ emotions for a safe driving experience.
Sleepiness and Emotion Detection with CNN and MediaPipe
According to research by Steinhauser et. al, inattention of the driver due to additional tasks, emotion changes, fatigue or eye movements played an important role in 78% of car accidents on the road. The National Highway Traffic Safety Administration estimates that at least 100,000 police-reported crashes are the direct result of driver fatigue each year. This equates to approximately 1,550 deaths, 71,000 injuries, and $12.5 billion in monetary losses. The goal of this SIR is to develop a system that detects and alerts unstable emotions and drowsiness during daily driving, in order to reduce that fatality on the road, saving dozens of lives. The SIR project monitors and analyzes driver’s facial features using real time video and alert drive when drowsiness or unstable mood is detected. The project, based on open-source code, has integrated libraries of face-meshing (MediaPipe), emotion detection, TensorFlow (CNN), and the eye-aspect-ratio algorithm we developed. It classifies multiple emotions and identifies drowsiness effectively. Future explorations can be in the area of automatic driver’s mode control. Emotion detection can serve as a regulator to adjust automotive informatics and climate systems to help stabilize drivers’ emotions for a safe driving experience.