Enhancing Driver Safety: Real-Time Eye Detection for Drowsiness Prevention Driver Assistance Systems ^†

Abstract

Drowsiness has become a significant contributing factor to traffic accidents in modern times, posing a major concern to society. Driver fatigue or sleepiness leads to decreased reaction time, diminished attention, and compromised decision-making abilities, thereby affecting the overall driving experience. This paper addresses this issue by proposing a drowsiness detection system based on image processing, utilizing a cascade of classifiers built on Haar-like features. The system effectively detects the eyes, allowing for determination of eye closure or opening, which serves as an indicator of driver drowsiness.

1. Introduction

Over the past few years, a significant number of electronic driving assistance systems have been developed and implemented with the primary goal of ensuring road safety, in direct response to the alarming prevalence of traffic accidents [1]. Traffic accidents are predominantly caused by driver errors and mistakes. In response to the alarming rise in fatal accidents, there has been an increasing global aspiration for a safer world, free from vehicle-related incidents. According to the Road Safety Report published by the World Health Organization, Malaysia has recorded one of the highest fatality rates, with 25 deaths per 100,000 people, surpassing the regional average of 17.9 deaths per 100,000 people [2]. With road security being an absolute necessity, the implementation of Advanced Driver Assistance Systems (ADAS) has become paramount.

Various types of equipment are utilized in ADAS, including physical sensors, locators, ultrasonic devices, phonic mixer devices, cameras, and night foresight tools. Algorithms are employed to ensure safety by considering factors such as traffic conditions, hazardous situations, and weather conditions. The primary objective is to avoid potentially dangerous situations. Additionally, an on-board driver assistant system plays a crucial role in monitoring driver attentiveness, alertness, and fatigue. These systems implement diverse sensing modalities, with computer vision emerging as a vital solution [3].

2. Drowsiness Detection System

Driver drowsiness detection stands out as a significant category among Advanced Driver Assistance Systems. This prominence is primarily attributed to the strong correlation between driver drowsiness and car accidents. The two primary causes of road accidents and the resulting financial losses are driving while drowsy or in a sleepy condition [4]. The detrimental impact of driver fatigue, resulting in a loss of concentration, significantly impairs the car driver’s ability to make timely and effective decisions [5]. As per the Road Safety Web Publication, around 20% of car accidents can be attributed to driver fatigue [6]. Consequently, the monitoring of driver fatigue contributes to enhancing both driver and vehicle safety.

In the last decade, researchers have dedicated efforts to develop driver drowsiness supervision systems. Various methods have been introduced, such as the feature-based approach, which involves analyzing the driver’s facial images to detect drowsiness indicators like eye blinks, yawning, or head movements. These techniques can be implemented through diverse approaches. Türkan et al. [7] successfully implemented edge projection with wavelet domain image classification for face detection. Fletcher et al. [8] introduced another technique based on the measurement of the percentage of eyelid closure, which calculates the ratio of frames with closed eyes to the total eye frames, providing an indication of drowsiness. Additionally, skin color can be employed as a feature in face detection. Alshaqaqi et al. [9] presented a system that utilized the color of human skin to identify regions containing faces. Additionally, modern systems have incorporated wearable smart glasses for the purpose of drowsiness detection. Chen et al. [10] and Chang et al. [11] proposed a fatigue–drowsiness detection system utilizing wearable smart glasses integrated with an in-vehicle smart system. This computer-based glasses system has the capability to detect the drowsiness or fatigue state of the driver. Viola and Jones [12] introduced Haar feature-based classifiers, which have become a prevalent technique for detecting facial and eye features in drowsiness detection systems due to their effectiveness and efficiency. Significant research efforts have been dedicated to the field of driver drowsiness detection, with the objective of uncovering the most effective and efficient system solutions.

3. Drowsiness System Development

The block diagram depicting the proposed system is presented in Figure 1. Raspberry Pi 3 serves as the main microcontroller for this system. The system is powered by a 5 V Micro USB connected to the car adapter. The Raspberry Pi NoIR Camera V2 is linked to the main controller via a camera serial interface. The main controller directly connects to the Touch Screen Display and active buzzer, which serve as system outputs. To store data, a 32 GB SD Card is utilized in this system.

3.1. Flowchart of the System

The process of the Eye Detection System for Driver Assistance begins by setting up and turning on the camera, as depicted in Figure 2 and Figure 3. Subsequently, the Haar algorithm method is loaded to detect the presence of the driver’s eyes. If the driver’s eyes are detected, Dlib’s facial landmark predictor is utilized to identify 68 salient points and draw a rectangular shape around the eyes to capture the eye area.

Next, the Eye Aspect Ratio algorithm is applied to extract the eye regions, which are then displayed on the LCD touchscreen. At this point, the user has the option to exit the system by utilizing the power-off button on the touchscreen, thereby ending the process. If the user chooses to continue using the device, the process restarts from point A, searching for the driver’s eyes using the Haar algorithm method.

During the sub-process labeled as B, the camera continuously monitors the driver’s eyes’ aspect ratio (EAR) and counts the frames if the EAR falls below the set threshold. If the count of frames is below the required threshold, the process returns to point A. However, if the count of frames reaches the specified consecutive frames threshold, the alarm is triggered. The process then loops back to point A again. This entire process continues until the device is turned off or shut down.

3.2. Electrical Wiring Connection

The connections between each component are illustrated in Figure 4.

4. Experimental and Data Analysis

A system prototype, depicted in Figure 5, has been successfully developed. It was designed as a portable device that can be easily installed in various vehicle models. The core concept of this system revolves around real-time video monitoring of the driver’s eyes using a camera, enabling reliable measurement of driver drowsiness levels.

4.1. Eye Aspect Ratio (EAR) Threshold

Figure 6 shows eye detection in the normal eye state and below the threshold conditions. Multiple threshold values ranging from 0.1 to 0.4 are collected. A blink is recorded when the eye aspect ratio drops below a specific threshold and subsequently rises above the threshold. No alarm detection occurs for thresholds 0.1 and 0.2. However, an alarm is detected at the preferred threshold of 0.3, as indicated in

Table 1. Threshold test results.

Threshold	Open	Close	Alarm
0.1	0.341	0.181	No
0.2	0.312	0.187	No
0.3	0.32	0.2	Yes
0.4	0.326	immediate	Yes

.

4.2. Detection Analysis

The detection analysis involved testing variable subjects and variable distances, as presented in

Table 2. Variable subject versus variable distance results.

Distance	Subject	Open	Close	Alarm Detection (Second)
40 cm	A	0.33	0.129	2.8
	B	0.232	0.1	0
	C	0.3	0.159	2
	D	0.246	0.084	0
	E	0.312	0.162	2.6
50 cm	A	0.365	0.2	2.69
	B	0.2	0.071	0
	C	0.32	0.129	2.21
	D	0.221	0.11	0
	E	0.32	0.181	2.61
60 cm	A	0.351	0.189	2.6
	B	0.243	0.093	0
	C	0.318	0.17	2.5
	D	0.237	0.08	0
	E	0.327	0.16	2.8

and

Table 3. Subject details.

Subject	Gender	Age	Race	Wearing Glasses
A	Male	39	Malay	No
B	Male	28	Indian	Yes
C	Male	27	Chinese	No
D	Female	35	Malay	Yes
E	Female	33	Malay	No

. Subject variables were based on gender, age, race, and whether the subjects were wearing glasses or not. According to Table 2, the optimum distance for the camera to capture the drivers was determined. However, for drivers wearing glasses, the detection time exhibited some errors due to reflections from the glasses. On the other hand, for subjects without glasses, the detection time was relatively consistent across different ages, genders, and races. A variation in eye aspect ratio was observed, primarily attributable to differences in eye sizes among the subjects.

In summary, the novelty of this paper lies in its extensive evaluation of the drowsiness detection system, taking into account variables such as gender, age, race, and glasses-wearing status. This contributes to a more comprehensive understanding of the system’s performance under diverse subject characteristics and enhances its applicability in real-world scenarios.

5. Conclusions

Our eye detection system developed for driver assistance has demonstrated its capability to promptly detect drowsiness, aligning with the objectives of this project. By leveraging the Eye Aspect Ratio (EAR), the system effectively distinguishes between normal eye blinking and drowsiness, thereby preventing the driver from entering a state of sleepiness. Through analysis, it was determined that the optimal distance for accurate detection falls within the range of 40 cm to 60 cm from the camera. However, it is worth noting that the system experienced challenges in accurately detection for subjects wearing glasses, likely due to reflections from the glasses. Interestingly, the analysis revealed that the races and ages of the drivers did not significantly impact the EAR settings.