Adaptive Control of Streetlights Using Deep Learning for the Optimization of Energy Consumption during Late Hours

Abstract

This paper presents an adaptive control scheme for streetlights by optimizing the energy consumed using deep learning during late hours at night. A city’s infrastructure is not complete without a proper lightening system for streets and roads. The streetlight systems often consume up to 50% of the electricity utilized by the city. Due to this reason, it has a huge financial impact on the electricity generation of the city. Furthermore, continuous luminosity of the streetlights contributes to the environmental pollution as well. Economists and ecologists around the globe are working hard to reduce the global impact of continued utilization of streetlights at night. In regard to a developing country which is already struggling to produce enough electrical energy to fulfill its industry requirements, proposing a system to lessen the load of the energy utilization by the streetlights should be beneficial. Therefore, an innovative and novel energy efficient streetlight control system is presented based on embedded video processing. The proposed system uses deep learning for the optimization of energy consumption during the later hours. Conventional street lighting systems consume enormous amounts of electricity, even when there is no need for the light, i.e., during off-peak hours and late at night when there is reduced or no traffic on the roads. The proposed system was designed, and implemented and tested at two different sites in Karachi, Pakistan. The system is capable of detecting vehicles and pedestrians and is able to track their movements. The YOLOv5 deep-learning based algorithm was trained according to the local requirements and implemented on the NVIDIA standalone multimedia processing unit “Jetson Nano”. The output of the YOLOv5 is then used to control the intensity of the streetlights through intensity control unit. This intensity control unit also considers the area, object and time for the switching of streetlights. The experimental results are promising, and the proposed system significantly reduces the energy consumption of streetlights.

1. Introduction

According to the Pakistan Economic Survey 2019–2020, Pakistan is one of the countries that not only generates the most expensive electrical energy by burning petroleum products, gases and coals; but its current transmission and distribution infrastructure is not sufficient to meet the peak residential and industrial demands [1]. Therefore, energy saving and optimization of energy consumption are essential for Pakistan and the current urban and suburban settlements of today’s world. To date, various approaches have been used for saving electrical energy, and one is implementing intelligent or smart control for streetlights. Since the beginning of the twentieth century, street lights have been a essential component of urban and suburban infrastructure around the world. They are the lifeblood of the city’s infrastructure and provide citizens with safe roadways; modern and pleasant public spaces; and increased security in homes, businesses and urban districts. The purposes of using streetlights are too many to count. However, they also utilize a significant amount of electricity. Streetlights are among the largest consumers of electricity and account for 40 to 50 percent of the usage of electricity in urban societies [2,3]. There are 326 million streetlights globally, and the number of streetlights will be increased globally to more than 361 million by 2029 [4].

In addition, the electricity generated for streetlights causes a significant environmental impact. Streetlights account for about 8% of environmental pollution due to greenhouse gas emissions [5,6,7]. Along with their environmental impact, the cost of their electricity has forced policy makers to implement solutions that reduce energy consumption. Therefore, the use of intelligent street lighting has grown to be an essential strategic element for social, economic and sustainable development. To date, numerous studies and efforts have been made to create intelligent streetlights with minimal electricity consumption [8].

The first and probably the most economical method is line control (LC). Line control uses astronomical clocks and power line regulators to control the power of streetlights [9]. The line regulator allows a group of streetlights to dim in the same way [10]. The second method used to control energy savings is the remote point-to-point approach or protocol (P2P). This approach is an extension of the line control method. It uses a power line communication (PLC) device with a line regulator to remotely control the intensity of each streetlight [9,11,12]. LC and P2P methods provide energy savings of up to 15% and 30%, respectively; however, both are static approaches based on constant settings [13,14].

The third method is a smart lighting control strategy that dynamically controls the electrical energy in a streetlight. The approach is derived from remote P2P control, and it incorporates the concept of “energy on demand” that is proportional to requirements such as pedestrian and vehicular traffic. The smart lightning is implemented using the Internet of Things (IoT). Smart lighting is an integral platform of smart cities and optimizes streetlight management and streetlight health, and reduces their power cost by 40% [15]. The smart approach using artificial intelligence or machine learning provides more options of lighting control and power optimization [16]. However, smart lighting also requires high initial costs due to additional infrastructure: hardware and software components, such as controllers, a remote management system and a high speed sensor network [9,14,17,18,19].

When reviewing the state-of-the-art street lighting illumination control, it has been noted that the measurement model of a streetlight control system is based on a passive infrared (PIR) sensor, an ultrasonic sensor or a radar sensor [20]. The first two are cost-effective solutions, but they also have serious accuracy issues. In addition, a sudden change in lighting can cause inconveniences to drivers and can lead to accidents. The accuracy of radar sensors, and therefore, driver comfort, is much better with PIR and ultrasonic sensors; however, it is also very expensive and only works at short distances [20,21]. Few studies reported the use of image sensors for the detection of moving traffic or pedestrians. However, all studies were based on offline videos and were static in nature. In this research, a dynamic streetlight control system was developed which process the video on site in real-time using a deep learning technique, which reduces the errors due to astronomical clock power regulations and/or remotely controlling the streetlight intensity.

Moreover, all these studies were conducted in first-world countries where sufficient infrastructure is already available to conduct and implement research and studies on smart lighting systems [14,22]. Additionally, their street lighting systems are based on standardized architecture [23], which makes the implementation much simpler and easier. In contrast, the street lighting system in Karachi, Pakistan is a mix of everything. There are old mercury bulbs and high-pressure sodium lamps. Several streetlights are outdated and inefficient, leading to increased energy requirements and regular maintenance [24]. Recently, LED lights were installed at various locations. The number of streetlights in Karachi is approximately in the thousands [25], but nowhere is there an intelligent street lighting system. The main hindrances in implementing the smart street lighting system are the technology as per local needs, high initial investment and inadequate knowledge on the likely benefits for significant energy savings that could be obtained by moving to smart lighting systems [22]. Billions of rupees are lost every year due to the lack of an intelligent street lighting system, as the streetlights burn with full intensity throughout the operation time window without any need or requirement. In conventional lighting systems, electrical energy is constantly squandered in lighting the entire road. Furthermore, not all roads are busy with vehicles or pedestrians during the whole operation time window.

This research work is part of a larger study and an extension of previous work [26]. This research focuses on a global smart energy efficient system that can improve energy consumption in streetlights using an embedded vision system and deep learning technique [26]. In it, a streetlight brightens up to full intensity only when it senses an approaching vehicle or pedestrian. This not only saves electricity, but also slows down global warming by reducing the demand for energy in streetlights. In this research paper, a novel energy-efficient street lighting intensity control system using embedded video processing is developed and presented. This novel system overcomes the limitations of previous research by eliminating the offline, static control and astronomical clock power regulation with an on-site, real-time, deep-learning-based video processing and decision system to control the light intensity of the streetlights. The development system ensures not only outstanding energy-saving levels, but also provides economic and environmental benefits.

This study discusses the financial and environmental advantages of replacing conventional streetlights with smart streetlights with intensity control based on traffic volume. A novel, real-time embedded video processing-based advanced control system was developed and implemented that focuses on energy saving and environmental pollution reductions, which could emphatically affect the general well-being. The novel control system adjusts the luminance of streetlights according to the pedestrian and vehicular traffic density on the road. The experimental setup shows that the optimal amount of energy saving can be achieved following seasonal daylight timing.

The organization of the rest of the paper is as follows. Section 2 describes the hardware components, software and algorithm that were used in this research work. Section 3 discusses the methodology for the prediction of the traffic and light control. Section 4 presents the experiments conducted and their results, specifying the details. Finally, Section 5 concludes the paper with a summary of the main results, and presents the future work.

2. The System’s Design

The proposed system was designed by keeping in mind the essential parameters of the streetlight intensity control. Among numerous parameters that may affect the performance of a streetlight system, the following parameters were selected:

• Type of light: The advancement in the streetlight system is mainly dependent on the energy consumption by the light source. In order to optimize the overall energy consumption, low power light emitting diode (LED)-based streetlights were selected.
• Traffic volume: Another major parameter to optimize the energy consumption is to control the intensity of the light. For this purpose, this research focused on adjusted the streetlight intensity based on the traffic volume at the experimental side.

Keep in mind that the designed system is divided into two sub-modules, i.e., a physical system and an algorithm.

2.1. Physical System

The physical system consist of a camera, an intensity control unit and a processing unit to handle the software and algorithms to detect the traffic. To capture the traffic data on the street, an Hikvision DS-2CD4A85-IZH UHD camera was used, which provides 4K resolution and H.264 video compression at 30 frames per second. The NVIDIA Jetson Nano was used as an onboard computational unit for processing live video streams. It is based on 64 bit quad-core ARM Cortex-A57 processor and was run at 1.4 GHz with a NVIDIA Maxwell GPU with 128 CUDA cores capable of 472 GFLOPS (FP16) [27]. With a tensor core unit for machine learning and deep learning, it provides sufficient computational power for onsite processing.

2.2. Algorithm Design

The algorithm is divided into two main parts, i.e., the flow control and the prediction model; the later is presented in Section 3 in detail. The flow control of the system is shown in Figure 1, The camera is connected with the Jetson Nano. Jetson Nano processes the video data and saves them in the database. After the addition to the database, the frames are extracted to implement the feature extraction on the raw data. These data are utilized for the training after pre-processing of the extracted frames. These frames are then used to estimate the number of vehicles and traffic flow. In the end, the traffic flow volume is then utilized to predict and control the streetlight intensity with the intensity controller.

3. Prediction Model—Road Traffic Forecasting

To predict road traffic, including pedestrians, a road traffic model has been derived that models the average number of vehicles passing through a point at any given time t on the road [28]. The model uses the annual average daily traffic flow X, which represents the daily average number of vehicles passing on a road based on measurement. Since the vehicular traffic moves in a specific direction, it can be defined using a weakly stationary stochastic time series model with the following:

$${\dot{X}}_t=c+\sum _{i=1}^p{\varphi }_i{\dot{X}}_{t-i}+{ϵ}_t$$

(1)

where $X_t$ represents the annual average daily traffic flow at time t. $0\le {\varphi }_i<1$ is the bounded model parameter, c is the constant and ${ϵ}_t$ is the bounded model uncertainties: $0<{ϵ}_t\le 1$. The pedestrian traffic model can be estimated using Equation (2).

$${\dot{\theta }}_t=\sum _{j=1}^p(1-{\Delta }_{ped}){\dot{\theta }}_{t-j}+{\sigma }_t$$

(2)

where ${\Delta }_{ped}$ is a percentage of the additional pedestrian traffic and defined as $0\le {\Delta }_{ped}<1$, and ${\theta }_t$ is the annual average daily traffic flow at time t, and ${\sigma }_t$ is the bounded model uncertainties as $0<{\sigma }_t\le 1$. Therefore, the total amount of road traffic at time t is given by Equation (3):

$${{\Gamma }_t={\omega }_t({\dot{X}}_t+\dot{\theta }}_t)$$

(3)

where $0\le {\omega }_t\le 1$ is a parameter that normalizes road traffic, where 0 means no traffic or pedestrians detected in video frames.

3.1. Street Lighting Control

This section describes the strategy that is used to control the streetlights. The proposed strategy is formed by the design criteria and lighting regulations for the assurance of both user road safety and energy savings. In general, the two different approaches are used to get the power for the lights follow.

3.1.1. Static Control Approach (SCA)

SCA is the most widely used approach because it is much easier to implement. SCA controls the intensity level of a streetlight according to constant parameters, such as on/off time periods as per the calendar, and keeps the intensity level at 100% for half the night and at 50% for the remaining half.

3.1.2. Smart or Adaptive Control

This control is an intelligent approach to control the intensity level of streetlights. It uses the energy on demand approach and provide optimal energy savings. The basic concept is to set the power level of the streetlight at time t as a function of road traffic (Equation (4)).

$$P_t=f\left({\Gamma }_t\right)$$

(4)

where $P_t$ is the power level normalized in [0, 1] to be set for the next time instance t, and ${\Gamma }_t$ is the road traffic at time t. In this research, an adaptive control scheme has been adopted due to the fact that the light requirement is based on the traffic volume rather than the duration of night. During the hours of 7–12 p.m., there is a great deal of traffic. Still, after 12 p.m., the traffic flow reduces, and thus, light being on consistently when there is no need leads to pure energy wastage. Therefore, it has been decided for our system that the light intensity should be a function of vehicle detection on the road after 12 p.m. Once the parameters are fixed, the conditions for the intensity control with reference to the traffic flow can be divided into following modes:

• Mode 1: No change in the intensity of streetlight: The first condition or mode for the intensity control is when there is no change in the traffic. This condition has two possibilities: either there was no traffic and there is no traffic, or there was traffic flow and there is traffic flow. In both conditions in Mode 1, there is no requirement to change the intensity of the light. If there is no traffic and there was no traffic, then the streetlight will remain off or at their lowest intensity. Similarly, when there is a flow of traffic and there was the same traffic flow, then there is no need to change the intensity of the streetlight. Therefore, this mode will not change the intensity of the streetlight but maintain the already set intensity of the streetlight.
• Mode 2: Change in the intensity of streetlights: The second condition for the intensity control is when there is change in the traffic flow. For a change of traffic flow there are two possibilities: either there was no traffic initially and the camera then detects traffic, or there was traffic flow and at next instance no traffic flow. In both conditions of Mode 2, there is significant analysis required to increase or decrease the streetlight intensity. This mode is thus required to have road traffic measurement for proper functioning. The road traffic measurement is discussed in detail in the upcoming section.

3.2. Road Traffic Measurement

The detection and tracking of vehicle in traffic were done using the YOLOv5 running on NVIDIA Jetson Nano, as shown in Figure 1. Directly from the HD camera video feed, image frames were extracted to develop a local database that was utilized for the training, object detection and tracking. The algorithm is capable of detecting and tracking multiple objects within an image frame in real-time. A model database was created using the MS COCO (Microsoft Common Objects in Context) and camera video feed data. Image frames were extracted from the cameras that are installed at the Faculty of Engineering, Ziauddin University and Dr. Ziauddin Hospital, North Nazimabad. Afterward, the dataset was divided into training and test sets and labeled using a labeling tool. Images with bounding boxes were labeled using labeling software, and an XML text file with the boxes’ coordinates for each image was structured into training and test sets in the database. The model was trained using the training set, and after processing the image frames, generated output matrix $\mathbf{O}$ defined as:

$$\mathbf{O}=(x_1\dots x_q,y_1\dots y_q,h_1\dots h_q,w_1\dots w_q,c_1\dots c_q)$$

(5)

where $x_i$ and $y_i$ are the coordinates of the i-th object; $h_i$ and $w_i$ are the height and width of the i-th object in pixels; and $c_i$ is the class label of the i-th object.

After detecting and recognizing the objects, it is essential to identify the stationary and non-stationary vehicles on the road, as vehicles parking on roads is very common. Thus, to determine if the vehicle is moving, YOLO is used with a combination of region-based techniques. An ROI is created, and the background model was developed for a user-generated region area, a region of interest (ROI). The essential step is to figure out how to transform the collected items into vehicle count data without detection and tracking. When a car passes across the ROI in subsequent frames, a portion of it is identified as a foreground object in each frame. The issue is determining how to count these items as single cars without repeating the process. To do so, each identified foreground region in each frame is assigned a number $N_{fv}$, where f and v, respectively, indicate the frame number and the vehicle label inside this frame (1:m). After that, each of these areas is compared to the regions identified in the preceding frame $N_{uf1}$, where u = 1:n.

The common area between two items in two successive frames is used to do this evaluation. If the shared area exceeds a certain threshold, these things are deemed part of the exact vehicle. In this scenario, the object in frame f uses the same vehicle label as the matched object in frame $f-1$, i.e., $N_{vf}=N_{uf}=1$, unless a new label is used. The vehicle number is raised by one. Vehicles are recognized as foreground items as they pass through this small region. Using the retrieved items, an efficient technique for counting cars was developed. There is no requirement for a tracking step, which is a significant distinction from previous methods. The obtained findings confirmed the suggested method’s outstanding performance, and it is shown in Figure 2.

In this work, firstly, the feature extraction technique was used to extract the necessary data from the frames. Next, the data labeling tool was used to label datasets. After labeling, the dataset was converted to the format required for deep learning and was used for convolutional neural network model training. After training, a trained model was used to predict vehicles in the live video feed. The developed model was designed and trained to classifies vehicles and other items. It also defines a region for object tracking to detect moving objects, and counts the number of vehicles on the road, including the identification of parked vehicles. The proposed system was used to control the power consumption of light based on detected vehicles to provide on-demand electrical energy and avoid wastage of electrical energy. In addition, the algorithm predicts the class using the features identified in the convolution process. The prediction depends on the number of classes that are defined during the training. The class prediction vector is defined as:

$$\mathsf{\Phi }=({\phi }_1,{\phi }_2,\dots ,{\phi }_k);0\le {\phi }_i<1$$

(6)

where ${\phi }_i$ is the probability of the i-th class and k is the total number of classes.

4. Results and Discussion

This section describes the experimental arrangements and their results in detail. An experimental setup was important for the verification of the proposed model’s efficiency and quantification of the energy savings in streetlights. There were two main parts to the experimental setup. First was the detection and tracking of vehicles using YOLOv5 and NVIDIA Jetson Nano in a live video stream. Finding vehicles at night is difficult and practical for controlling streetlights. The results of vehicle detection and tracking are presented and discussed in detail. The second part was to calculate energy savings through the proposed approach. Finally, conclusions are drawn from the results obtained.

4.1. Experimental Setup

The experimental setup was placed at two different areas after careful consideration of the traffic flow. The first site was the ZU site, which is a road in front of Ziauddin University, Faculty of Engineering, Science, Technology and Management (ZFESTM), Block B, North Nazimabad, Karachi, whereas the second site is labeled as the SSUET site, which is an entrance road of Sir Syed University of Engineering and Technology, Gulshan-e-Iqbal, Karachi. The experimental setups on the selected sites are shown in Figure 3. The cameras are mounted on the side of the road. The reasons for choosing both sites were easy accessibility and there being nominal traffic flow on weekdays and weekends. Using Google Maps, we point out that both roads are about 1000 m long, and both roads have an average of 30 m between the streetlights, which provides excellent comparisons for the study of lighting schemes, as both sites have major traffic flow not only at day time but at night as well.

4.1.1. Object Detection Using YOLOv5

The availability of datasets is essential for the detection and tracking of objects. However, no standard road video datasets are available for the local vehicles and roads of Karachi. Figure 4 shows the setup used to record test videos near the ZUFESTM, North Nazimabad site. The videos were recorded with different illumination conditions and shadow effects and include different types of vehicles (bicycles, motorcycles, rickshaws, cars, vans, trucks and more). In addition, the test videos have complex backgrounds with surrounding buildings, moving plants, trees, birds and more. YOLOv5 is based on the YOLO detection architecture and uses the best algorithm optimization strategies in the field of object detection for neural networks [29]. The YOLOv5 architecture was modified for the traffic detection application and is shown in Figure 5. It consists of three parts: (1) Backbone: CSPDarknet (cross stage partial Network), (2) Neck: hidden layer, and (3) Output Layer. The data is the first input to CSPDarknet for feature extraction, and then fed to hidden layer for feature fusion. Finally, output Layer outputs the detection results (class, score, location, size). To train the YOLOv5 model, the video streams were converted into the frames, and then frames were converted into annotations. LabelImg and Roboflow tools were further used for converting the annotations file into YOLOv5 format. After, annotations were tested, and dataset augmentation was done for the purposes of scaling, color space adjustments and rotation.

For the evaluation, the precision and recall for every class were tested, which are defined as:

$$Precision=\frac{TP}{TP+FP}$$

(7)

and

$$Recall=\frac{TP}{TP+FN}$$

(8)

where $TP$ represents true positives, which define the correct detection of objects; $FN$ represents false negatives, and they are objects which were present but not detected by the algorithm; and $FP$ represents false positives, which are invalid or incorrect detections of objects. Precision measures the percentage of accurate hits or predictions and is defined as the proportion of true positives of all positive results (both true positives and false positives). The recall defines the percentage of total relevant results correctly classified by the network. To determine if the prediction is a true positive or a false positive, intersection over union (IoU) has to be measured. The intersection over union (IoU) assesses the overlap area between the predicted bounding box and the actual item’s ground truth bounding box in object detection. Detection can be categorized as correct or wrong by comparing the IoU to a specified threshold. The accuracy of the algorithm is defined by the mAP, which is the proportion of true results (both true positives and true negatives) among the total number of cases examined. When computing mAP, intersection over union (IoU) is utilized. It is a value between 0 and 1 that indicates how much the anticipated and ground truth bounding boxes overlap. As each value of the IoU threshold yields a distinct average accuracy (AP) measure, this value must be specified. The result of evaluation of YOLOv5 model on NVIDIA Jetson Nano is shown in Figure 6 and summarized in

Table 1. Yolov5 training parameters.

Label	Image Size	Batch Size	Recall	Precision	mAP 0.5	mAP 0.5:0.95
all	640 × 640	16	0.98	0.987	0.959	0.756
5	640 × 640	16	0.978	1	0.965	0.879

.

The first column in Table 1 represents the labels of the object which we trained on 80 classes of object on the coco dataset, and also we customized our training data with selected objects such as cars, trucks, motorbikes, bicycles and buses. The second column represents the image size of images during the training. Third column is for batch size for the model. Fourth and fifth columns represent the recall and precision. Sixth and seventh columns have average precision for our model.

4.1.2. Results of the Loss during the Training

A 10,000-image dataset was used with the YOLOv5 model. Figure 7 depicts the projected bounding box loss, a cell holding an item and the model’s class loss. The findings of YOLOv5 are shown in blue.

Table 2. Classification performance analysis.

Object	Success Result	Car	Motorbike	Bus	Truck	Bicycle
Car	98.10%	-	-	-	1	-
Motorbike	94.84%	-	-	-	-	2
Bus	91.72%	-	-	-	-	-
Truck	93.72%	1	-	-	-	-
Bicycle	89.93%	-	2	-	-	-

shows the success and misclassification on the dataset, and it shows the success rate that was achieved on the customized dataset of vehicles. It also shows the percentage of misclassified instances in the analysis conducted. The results of classification are shown in Figure 8.

After training, NVIDIA Jetson Nano is used for detection, identification and tracking of road vehicles in a video stream using the YOLOv5 algorithm. The main focus was to locate vehicles in live video frames. Figure 9 shows the detection of vehicles in a night video stream. Despite poor lighting and night video, YOLOv5 was able to identify and track the vehicle.

4.2. Energy Saving Results

After identifying the vehicles in the video frame, the next step is to control the intensity of streetlights using the proposed approach and provide on-demand electrical energy. Figure 10 and Figure 11 show the statistical weekly vehicle profiles of both SSUET and ZU sites. It can be seen that these road traffic profiles are very similar over a week. From 11 p.m. in the evening, the number of vehicles on the road starts decreasing, and the vehicle counts are very low after 2 a.m. Starting at 7 a.m, the vehicle counts gradually increase at both sites. After 9 am, both sites have significant traffic flow.

Normally, streetlights have to work for 10 to 12 h daily, which causes significant electricity usage using the traditional method. The proposed adaptive control system operates streetlights at different intensities for the optimization of energy consumption, depending on the vehicles on the road, at two sites. Figure 12 shows the average intensity level and power consumption of the streetlights during the twelve-hour operational cycle from 6 p.m. to 6 a.m. Due to the low availability of ambient light at 6:00 p.m, the intensity level or power consumption of streetlights was set to 80%. Power consumption is 112 watts with 80% intensity. There are working hours till 12 o’clock at night, and due to the large number of vehicles on the roads, the lights are run at maximum intensity and use 140 watts. Between 12 noon and 2 a.m, the average power consumption was reduced by 70% due to reduced traffic flow and used 98 watts. Due to less traffic on the road, between 2 a.m. and 4 a.m., the average power consumption was further reduced by 60%, and 84 watts was used. In addition, from 5 a.m. to 6 p.m., the average power consumption was further reduced by 50%, and only 70 watts was used.

Table 3. Electrical power consumption using the conventional method and proposed approach.

Parameters	Conventional Method Using Sodium Lights	Conventional Method Using LED Lights	Proposed Approach Using LED Lights
Operational Hours	12	12	12
Power Rating (W)	300	140	140
Power Consumption (W)	3600	1680	1232
Units/Day	3.6	1.68	1.23
Units/Year	1314	613	449
Price/Year/Light (PKR)	24,800	11,590	8830
Percentage of Saving	−113%	0%	31.25%

summaries the power consumption using the conventional and proposed approaches. Let us assume that the streetlights are in operation for twelve straight hours. A typical sodium vapor lamp used for streetlights is generally 250–300 watts. A single 300 watt sodium vapor lamp consumes 3600 watts in twelve hours; that is equivalent to 3.60 units per day and 1314 units per year. The LED lights used in this project are 140 watts and consume approximately 1680 watts in twelve hours, which is equivalent to 1.68 units per day and 613 per year. According to the data acquired using the experimental setup, the typical daily electrical consumption would be approximately 1232 watts, and that is equivalent to 1.232 units per day and 449 units per year for a single streetlight. The block price for the 449 units from K-Electric considering the on peak and off-peak hours is approximately Rs. 8835 for a year. For the conventional approach, the annual cost of each streetlight is approximately 24,500, or 11,950 for a sodium vapor lamp or LED. There is such a huge price difference between using the conventional approach and using the proposed approach that a minimum of Rs 3000 can be saved per year on a single streetlight. According to CBC (Cantonment Board Clifton, https://cbc.gov.pk/en/street-light, accessed on 3 August 2022), there are approximately 13,000 streetlights installed on their premises using the conventional mode of operation, and they are in operation for 12 h a day. Let us assume that the proposed approach is integrated with the existing architecture of CBC; it would save approximately Rs. 3000 per light, and according to data, 13,000 transforms into Rs. 3000 per year: 39 million per year in total.

Hence, the designed system and proposed approach show remarkable energy savings, and if only implemented in the CBC could save around 39 million rupees per year. The proposed technique reduces the inaccuracies due to the static and offline approach of the previous system, but also, a few directions may be focused on further enhance the adaptability of the system. In this regard, we should consider reducing the system’s cost by developing an embedded system instead of utilizing expensive off-the-shelf components. Furthermore, the system may be incorporated with obstacles avoidance techniques to cater to the issues that may arise due to bad weather and implementation of the system at a large scale. However, the only sites so far were selected carefully so that no obstacle avoidance techniques were required. A technique such as “automatic object removal with obstructed façades completion using semantic segmentation and generative adversarial inpainting” [30] may be utilized for future enhancement of the system.

The proposed system does not involve the installation of any sensor other than a camera, which is already installed on most streets to control and monitor the traffic. Therefore, the developed technique may utilize the already available hardware (i.e., camera) and process the video in real-time to control the intensity of the streetlights based on the traffic flow detected using the deep learning algorithm (YOLO v5). This gives the proposed system an added advantage over the available traffic systems and modules available in the market.

5. Conclusions

The consumption of electricity by the streetlights of a city makes up as much as 50% of the total. This amount of electricity consumed by the streetlights not only burdens the economy of the city, but also produces a reasonable impact on the environment by emitting carbon dioxide. The aim of this research was to develop an adaptive control system that optimizes the utilization of streetlights and thereby decreases the energy consumed. This novel approach of an efficient streetlight energy utilization system is based on embedded video processing incorporated in the NVIDIA standalone multimedia processor “Jetson Nano”. The Jetson Nano shows remarkable results of processing real-time video. This video sequence is fed into the YOLOv5 algorithm, which is trained on the local dataset to optimize performance. YOLOv5 detects and identifies the moving person or vehicle. After the detection of the object, the signal is transmitted to an intensity control circuit to optimize the streetlight intensity according to the modes characterized for the intensity control. The results show that the traditional streetlight based on LED utilizes 613 units per year. After the implementation of the adaptive control scheme, the consumption is reduced to 449 units in one year, and about 164 electrical units are saved in a year for one light. However, reduction in the energy consumption may also vary depending on the location, traffic flow and light source. The future directions may involve reducing the costs of installation and essential components to provide return on investment as early as possible. Other dimensions for the expansion of the research will be to introduce dimming profiles and predictive techniques on weekdays and weekends. Furthermore, multi-sensory strategies using LiDAR could be a potential area of research. Overall, the system developed and introduced in the research has shown promising results and is able to contribute to the economic and ecological improvement of the city, and in turn aid in the development of the country. Furthermore, it may be concluded that the proposed system might easily be implemented in the city, as cameras are mounted on most of the highways for speed controlling; the same cameras may be utilized to implement the proposed system without adding any hardware installation cost.

6. Patents

A patents resulting from the work reported have been submitted for approval.