Event Title

Sleepiness and Emotion Detection with CNN and MediaPipe

Session Number

Project ID: CMPS 10

Advisor(s)

Victor Elarde; Northrop Grumman

Tao Zhang; Northrop Grumman

Discipline

Computer Science

Start Date

20-4-2022 10:05 AM

End Date

20-4-2022 10:20 AM

Abstract

According to research by Steinhauser et. al, inattention of the driver due to additional tasks, emotion changes, fatigue or eye movements played an important role in 78% of car accidents on the road. The National Highway Traffic Safety Administration estimates that at least 100,000 police-reported crashes are the direct result of driver fatigue each year. This equates to approximately 1,550 deaths, 71,000 injuries, and $12.5 billion in monetary losses. The goal of this SIR is to develop a system that detects and alerts unstable emotions and drowsiness during daily driving, in order to reduce that fatality on the road, saving dozens of lives. The SIR project monitors and analyzes driver’s facial features using real time video and alert drive when drowsiness or unstable mood is detected. The project, based on open-source code, has integrated libraries of face-meshing (MediaPipe), emotion detection, TensorFlow (CNN), and the eye-aspect-ratio algorithm we developed. It classifies multiple emotions and identifies drowsiness effectively. Future explorations can be in the area of automatic driver’s mode control. Emotion detection can serve as a regulator to adjust automotive informatics and climate systems to help stabilize drivers’ emotions for a safe driving experience.

Share

COinS
 
Apr 20th, 10:05 AM Apr 20th, 10:20 AM

Sleepiness and Emotion Detection with CNN and MediaPipe

According to research by Steinhauser et. al, inattention of the driver due to additional tasks, emotion changes, fatigue or eye movements played an important role in 78% of car accidents on the road. The National Highway Traffic Safety Administration estimates that at least 100,000 police-reported crashes are the direct result of driver fatigue each year. This equates to approximately 1,550 deaths, 71,000 injuries, and $12.5 billion in monetary losses. The goal of this SIR is to develop a system that detects and alerts unstable emotions and drowsiness during daily driving, in order to reduce that fatality on the road, saving dozens of lives. The SIR project monitors and analyzes driver’s facial features using real time video and alert drive when drowsiness or unstable mood is detected. The project, based on open-source code, has integrated libraries of face-meshing (MediaPipe), emotion detection, TensorFlow (CNN), and the eye-aspect-ratio algorithm we developed. It classifies multiple emotions and identifies drowsiness effectively. Future explorations can be in the area of automatic driver’s mode control. Emotion detection can serve as a regulator to adjust automotive informatics and climate systems to help stabilize drivers’ emotions for a safe driving experience.