Tree Seedlings Detection and Counting Using a Deep Learning Algorithm

Abstract

Tree-counting methods based on computer vision technologies are low-cost and efficient in contrast to the traditional tree counting methods, which are time-consuming, laborious, and humanly infeasible. This study presents a method for detecting and counting tree seedlings in images using a deep learning algorithm with a high economic value and broad application prospects in detecting the type and quantity of tree seedlings. The dataset was built with three types of tree seedlings: dragon spruce, black chokeberries, and Scots pine. The data were augmented via several data augmentation methods to improve the accuracy of the detection model and prevent overfitting. Then a YOLOv5 object detection network was built and trained with three types of tree seedlings to obtain the training weights. The results of the experiments showed that our proposed method could effectively identify and count the tree seedlings in an image. Specifically, the MAP of the dragon spruce, black chokeberries, and Scots pine tree seedlings were 89.8%, 89.1%, and 95.6%, respectively. The accuracy of the detection model reached 95.10% on average (98.58% for dragon spruce, 91.62% for black chokeberries, and 95.11% for Scots pine). The proposed method can provide technical support for the statistical tasks of counting trees.

1. Introduction

Woods play a vital role in our lives as they are an essential source of oxygen and food for all living organisms on earth. Moreover, they provide various ecosystem services, combat global warming, and keep the air clean and healthy by taking carbon dioxide (CO₂) out of the air and releasing oxygen [1]. Hence, tree counting can have numerous benefits, and it plays a vital role in many environmental protection applications, forest resource management, crop yield estimation, and agricultural planning. Thus, keeping track of the number of trees in an area could be required for effective management and quantitative analysis [2]. The traditional tree counting methods, which depend on human visual inspection, require expensive feature engineering and are considered time-consuming and labor-intensive. Recently, the development of “precision forestry” [3] has required high technologies and analytical tools to support forest management and leverage conservation and use of forest resources [4]. Owing to the increasing evolution of model structure and the continuous updating of GPU hardware, deep learning has been vigorously developed and has shown significant advantages, recommending it as the mainstream method in the object detection field [3]. Recently, deep learning algorithms have been widely used to determine the height and the number of trees, demonstrating its outstanding potential for counting trees [5,6]. Yuan et al. used a binocular camera combined with YOLOv4 to count and measure the height of tree saplings [7]. Wu et al. [8] proposed a large-scale, real-time, and cross-regional oil palm tree detection (CROPTD) method containing a local and a global domain discriminator generated by adversarial learning. To highlight more transferable regions, they proposed the local attention that tells the neural network where are there more transferable regions. The CROPTD approach was evaluated on two large-scale high-resolution satellite images in Peninsular Malaysia. The CROPTD approach improves the detection accuracy by 8.69% in terms of the average F1-score compared with Faster R-CNN. Moreover, CROPTD performs 4.99–2.21% better than the other two domain adaptive object detection approaches.

Alburshaid et al. [9] also used high-resolution satellite images as a primary data source and presented an MRCNN model to recognize, count, and categorize date palm trees in the Kingdom of Bahrain. The model was tested on the entire land region of Bahrain’s Northern Governorate, which encompassed around 175 square kilometers. As a result, the precise positions and quantities of palm trees across the Northern governance were detected and kept in the agricultural geodatabase; the obtained result can assist in estimating the yearly yield for date palm trees in the Kingdom of Bahrain. Htet et al. [10] presented palm tree classification and counting using remote sensing drone video and the mask R-CNN algorithm with a retuning hyperparameter strategy. The tested area was comprised of the upper and delta coastal area of Myanmar, and over 12,000 aerial images were taken by drone for three types of palm trees (toddy palm, coconut palm, and palm oil); the research result exhibited 80–95% validation accuracy on each palm class. It is deduced that tuning learning will improve the performance of the local palm tree classification approach and can accurately detect and count toddy palm trees.

Recently, many researchers have been focusing on using remote sensing in agriculture due to the benefits of unmanned aerial vehicles (UAVs) [11], which can obtain high-resolution images that are less expensive than other satellites and aircraft that require high altitudes and other capabilities. In contrast, UAVs can fly at lower altitudes, facilitating the easy acquisition of clear images [12]. Csillik et al. [13] proposed a method to detect citrus trees from UAV images by a simple convolutional neural network algorithm. The classification approach, which was improved by superpixels that were derived from a simple linear iterative clustering algorithm, has achieved an accuracy of 94.59% and a recall of 97.94%. Osco et al. [14] presented a convolutional neural network approach to estimate the number of citrus trees in highly dense orchards from UAV multispectral images. The dataset was comprised of 2389 images that were acquired using a multispectral camera with four bands; green, red, red-edge, and near-infrared bands. The result showed high precision (0.95) and low MAE (2.05) compared with Faster R-CNN and RetinaNet algorithm, proposing this approach as a satisfactory and effective strategy to replace the traditional visual inspection method for counting the trees. Zheng et al. [15] used computer vision technology to process and analyze visible light images that were obtained by the UAVs in a citrus planting area in Xinhui District of Jiangmen City- China. Their research aims to detect citrus trees in the study area and count the mature trees and saplings. The obtained accuracy rate of the experiment reached more than 95%, which can meet the requirements of fast and accurate counting and planting situation of citrus trees in a planting area at low cost.

Furthermore, due to the economic value and ecological impacts of coconut trees, it is vital to detect and monitor them. Thus, many research papers reported on detecting and counting coconut trees [16,17,18]. Zheng et al. presented a method to detect and count the coconut trees from high-resolution satellite images that were acquired in Tenarunga island in the Pacific Ocean from Google Earth Map [19]. There were three major procedures that were applied: a multi-level region proposal network (RPN), feature extraction, and a large-scale detection workflow. The results showed a high average F1-score of 77.14% that outperformed pure Faster R-CNN with +1.33% in the average F1-score.

In summary, it is more practical to use deep learning algorithms to determine the number of trees because it is a fast, low cost, and effective tool [20]. Due to the increase in world population, there is an increasing demand for food and an urgent need to increase tree planting, which is required globally to maintain the water cycle, sequestrate carbon, conserve soils, and protect the continued existence of humans and animals. Moreover, the depletion of forestry resources dramatically is also increasing. Thus, in the agriculture sector, intelligent farming techniques and automated tree-counting approaches play an essential role in the crop management system, achieving rapid and accurate management of forestry resources and empowering the development of many economies worldwide [21]. Moreover, counting tree seedlings helps monitor natural succession and support conservation efforts of forestry resources.

This paper presents a new tree seedlings detection method using YOLOv5 with a dataset containing 660 images of three different types of tree seedlings. The dataset images were taken via drone and mobile phone. Then, data augmentation was used to obtain a large amount of data to train the detection model. Finally, the manual counting results are compared with the system. The correctness is evaluated by comparing the manual count results with the system count results. Our detection approach result is satisfactory and is an effective method to replace the traditional human visual inspection method to determine the number of tree seedlings in an area. The proposed method overcame the difficulties of the traditional tree seedlings quantity survey that needs staff to work in the field to obtain information and data, is humanly infeasible, requires high labor costs, and has small effective coverage.

2. Materials and Methods

2.1. Dataset

2.1.1. Data Collection

In this study, images of three types of tree seedlings (dragon spruce (Picea asperata), black chokeberries (Aronia melanocarpa), and Scots pine (Pinus sylvestris)) were collected and analyzed. The images were captured with various angles that were selected at different shooting distances. In total, the original images were 660, including 60 for dragon spruce, 300 for black chokeberries, and 300 for Scots pine. Furthermore, the outdoor environment factors were also considered during the image capture. The image shooting was done on a sunny day with natural light to ensure the precision of shooting images. The RGB images of the three types of tree seedlings are shown in Figure 1. The data were augmented via 13 methods of the major data augmentation methods. The details of the tree seedling dataset are shown in

Table 1. Information about the collected dataset.

Plant	Captured Images Area	Captured Images Way	Number of Captured Images
Dragon spruce	Tianheng mountain	DJI Inspire1 UVA	60
Black Chokeberries	Tianheng mountain	Mobile phone	300
Scots pine	Acheng district	Mobile phone	300
Total			660

.

The dataset was acquired from different places, as follows. Dragon spruce images and black chokeberries images were captured in Tian Heng Mountain Harbin city, Heilongjiang Province-China, and Scots pine images were captured in the Acheng district, Harbin city, Heilongjiang Province-China.

The dataset was acquired in two ways. First, dragon spruce images were taken by DJI Inspire1 professional drone with a wheelbase of 559–581 mm and a maximum flight time of 18 min. Its weight is slightly less than 7 lbs (2935 g), and it flies at a speed of 72 feet (22 m) per second. The UAV has a 12.4 million FC350 camera and SONY EXMOR 1/2.3 sensor, which can take a single shot, multiple continuous shooting, and a maximum 4096 × 2160 p resolution HD video recording. The tilt photography scheme is performed in one ortho and four oblique directions. Second, black chokeberries and Scots pine images were taken by a Huawei Enjoy 20 Pro mobile phone equipped with an Octa-core CPU (4 × 2.0 GHz Cortex-A76 & 4 × 2.0 GHz Cortex-A55) and a triple main camera (48 MP, f/1.8, 26 mm (wide), 1/2.0”, 0.8µm, PDAF), (8 MP, f/2.4, 120° (ultra-wide)), (2 MP, f/2.4, (macro)) and 4000 × 1080 p resolution video recording.

2.1.2. Data Preprocessing

After the data collection, the obtained data were pre-processed as follows. Firstly, we selected the images with the best shooting angle and cut their useless edges. Secondly, we modified the resolution and the brightness of the images. Thirdly, we delineated the bounding boxes of the seedling targets in the images and classified the categories to enable manual annotation of the seedlings. Then, 660 images were labeled using the image annotation software ‘LabelImg’ tool based on the smallest surrounding rectangle of a tree seedling to ensure the rectangle covers as small of a background area as possible. After data labeling, the position and classification of the tree seedlings were marked, resulting in JSON annotation files, and then they converted to txt files. Txt files contain information on object class, object coordinates, and the height and width of the labeling frame. Figure 2 shows the manual annotation of the tree seedlings using the ‘LabelImg’ tool, and Figure 3 shows the three types of tree seedlings after preprocessing.

2.1.3. Data Augmentation

Data augmentation is a technique of increasing the limited data to be significant, meaningful, and more diverse. Data augmentation prevents overfitting and improves the accuracy of the detection model [22]. To further enhance the model’s performance, it is necessary to use data augmentation methods to expand the sapling dataset. Referring to related literature work, the data in this study were augmented via 13 strategies of the major data augmentation methods, such as flipping, rotating, flipping and rotating at the same time, adjusting brightness, and flipping and adjusting brightness at the same time. Examples of the applied data augmentation are shown in Figure 4. The data augmentation process was applied after the data labeling process. Thus, images were augmented simultaneously with the label files. Therefore, the dataset was divided into two datasets in the ratio of (8:2): the training dataset and the validation dataset. The augmented training dataset contained 7000 images, and the augmented validation dataset contained 2000 images.

2.2. Methods

2.2.1. YOLOv5 Network Architecture

YOLOv5 is a single-stage object detector with four versions: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x [23,24]. YOLOv5x has the highest detection accuracy and the largest width and depth, whereas YOLOv5s is considered the fastest and smallest width and depth model among other models. All versions of the YOLOv5 network have the same network architecture, but the difference is in model widths and depths. The main architecture of YOLOv5’s four versions consist of three main parts: model backbone, model neck, and model head. YOLOv5 was designed to extract high-level features, reduce the processing time of the models, and preserve high accuracy [25].

In this paper, YOLOv5s was selected for tree seedlings detection not only due to its eligibility and fitness to effectively realize the requirements of tree seedlings detection task but also because it can be well deployed in agriculture and forest fields. YOLOv5s uses feature pyramids, which help the model to identify and match different sizes and scales of the same type of object [26]. Thus, YOLOv5s can detect different sizes of tree seedlings. Moreover, the feature map at the top layer of the pyramid can also have the advantage of the rich location information that is brought by the bottom layer and accordingly improves the model’s accuracy [27]. Figure 5 shows the structure of the YOLOv5s.

The model backbone of YOLOv5s is mainly used to extract essential features of the input image. It primarily consists of a focus module (Focus), standard convolution module (Conv), cross stage partial network module (C3), and spatial pyramid pooling module (SPP). The input RGB image is divided into four parts and concatenated by the focus module for better extracting the features during down-sampling. Then, the four parts of data are spliced in the channel dimension by the Concat module and then convolved to obtain the down-sampling feature map. The focus module increases the channel dimension and reduces the width and height of the feature map, improving the network’s performance by lowering FLOPS (floating point operations per second) and increasing speed. Figure 6 shows the slicing operation of the Focus module. The Conv is the basic convolution module of YOLOv5s that performs convolution operation, normalization, and activation operations for feature extraction. The C3 module is a designed module based on cross stage partial network, which is used to enhance the gradient values of backpropagation between layers and improve the learning ability of the model. Figure 7 shows the structure of the C3 module. The SPP module improves the receptive field of the network by converting the arbitrary feature map into a fixed-size feature vector. It mixes and pools spatial features through three parallel max pooling layers and then concatenates them [28]. Figure 8 shows the structure of the SSP module.

The model neck of YOLOv5s is composed of a series of path aggregation networks (PANets) [29], which can obtain FPN (feature pyramid networks) by assembling feature maps from different stages of the model backbone. The critical role of FPN is to enhance the semantic information by performing continuous down-sampling, which transfers the strong semantic features from the top feature maps into the lower feature maps. Whereas the PANets improve the accuracy of localization signals in lower layers by moving strong localization features from lower feature maps into higher feature maps, producing a noticeable enhancement of the location accuracy of the object [30]. The PAN and the FPN jointly strengthen the feature fusion capability and help the model to identify and match the different sizes and scales of the same target object.

The model head of YOLOv5s is the final detection part of the model. It is composed of three detection layers that have different size feature maps. Moreover, it has been designed based on PANet, which is same as that of YOLOv3 and YOLOv4. Finally, the corresponding anchor boxes are applied on the final feature maps of the three scales to generate the final output vectors with the position of the predicted bounding boxes and the categories of the target objects in the original image and label them.

2.2.2. Experiment Environment

In this research, the Ubuntu 20.04 operating system was the environment for this experiment. The training and testing processes of the model were run under PyCharm 2020.2.3 framework with built-in Python version 3.6.5 using a server with an Intel(R) Xeon(R) Gold 6128 CPU at basic frequency 3.40 GHz; 64 GB RAM and three NVIDIA GeForce GTX 1080 Ti 12 G GPU. All the codes and other required libraries that were utilized in this experiment were in Python language. The hardware and software parameters of the experiment environment are shown in

Table 2. Information about Hardware and Software experimental environment.

Name	Parameter
Experimental environment	server
Operating system	Ubuntu 20.04
CPU	Intel(R) Xeon(R) Gold 6128
GPU	NVIDIA GeForce GTX 1080 Ti
Deep learning framework	PyTorch 1.7.0
Language	Python 3.6.5
Deep Learning model	YOLOv5s model

. The specific training parameter configurations of the tree seedlings detection model are shown in

Table 3. Training parameter values.

Parameters	Optimization Algorithm	Learning Rate	Epoch	Batch Size	Image Size
Value	SGD	0.001	400	2	416 × 416

.

2.2.3. Evaluation Indicators

To evaluate the performance of the tree seedlings detection model, we used the following evaluation indicators: precision (P), average precision (AP), recall (R), F1_score, accuracy and the mean average precision (mAP), which are defined in the Equations (1)–(6), respectively.

$$\mathrm{Precision}=\frac{TP}{TP+FP}$$

(1)

$$\mathrm{Recall}=\frac{TP}{TP+FN}$$

(2)

$$\mathrm{Accuracy}=\frac{TP+TN}{TP+FN+FP+TN}$$

(3)

$$\mathrm{F}1\_\mathrm{score}=\frac{2PR}{P+R}$$

(4)

$$\mathrm{AP}={\displaystyle {\int }_0^1}P\left(R\right)dR$$

(5)

$$\mathrm{mAP}=\frac{1}{N}{\displaystyle {\sum }_{i=1}^N}A{P}_i$$

(6)

where TP (true positive) refers to the number of correctly identified tree seedlings, FP (false positive) refers to the number of incorrectly identified tree seedlings, and FN (false negative) refers to the number of unidentified tree seedlings. The precision represents the ratio of correctly identified tree seedlings to all identified tree seedlings, which are correctly and incorrectly identified. In contrast, the recall represents the ratio of correctly identified tree seedlings to ground-truth tree seedlings in the dataset.

3. Results

The model was tested with 30 images of dragon spruce tree seedlings, 50 images of black chokeberries tree seedlings, and 50 images of Scots pine tree seedlings. The specific detection results of the model before data augmentation are shown in

Table 4. The detection results of the model before augmentation.

Plant	TP	FN	FP	P (%)	R (%)	F1_Score (%)	Accuracy (%)	mAP
Dragon spruce	319	170	2	99.38	65.24	78.77	64.97	87.48%
Black Chokeberries	886	224	113	88.69	79.82	84.02	72.44	88.37%
Scots pine	329	29	20	94.27	91.90	93.07	87.04	93.24%

. The particular detection results of the model after data augmentation are shown in

Table 5. The detection results of the model after augmentation.

Plant	TP	FN	FP	P (%)	R (%)	F1_Score (%)	Accuracy (%)	mAP
Dragon spruce	485	4	3	99.18	99.18	99.28	98.58	89.87%
Black chokeberries	1050	60	36	96.69	94.59	95.63	91.62	89.19%
Scots pine	350	8	10	97.22	97.77	97.49	95.11	95.68%

. Examples of the detection results are shown in Figure 9. As expected, after data augmentation, the model performance was more reliable, and the accuracy of the detection model was improved.

For the dragon spruce tree seedlings, their morphological characteristics with a pyramidal shape and dense branches led to high detection accuracy. However, some detection is still missed, mainly concentrated in the places where the dragon spruce tree seedlings are relatively dense. In addition, the case of two overlapping tree seedlings will be regarded as one, and the tree seedlings that overlap with another type of tree could be missed from detection. In the case of black chokeberries tree seedlings, the missed detection is mainly concentrated in the places where the difference in color and shape of the target and the background is relatively slight, making the model recognize the target as the background. Moreover, the tree seedlings on the edge of the image could be missed from detection. For Scots pine tree seedlings, the more complex crown shape and structure made it challenging to separate the crown of the tree seedlings. Therefore, the model could not estimate tree seedling structure correctly. Moreover, the textured and cluttered backgrounds majorly affect the model’s performance. The undetected tree seedlings are labeled by a red ellipse in Figure 10.

4. Conclusions

This study presents a method for detecting and counting three types of tree seedlings based on the YOLOv5 object detection network. The data were augmented via 13 data augmentation methods to improve the accuracy of the detection model and prevent overfitting. The detection model was trained with 7000 images and validated with 2000 images. The optimal model was obtained at 400 epochs and the average detection time of dragon spruce, black chokeberries, and Scots pine tree seedlings was 0.695, 0.713, and 0.728 s, respectively. The detection of dragon spruce tree seedlings exhibited the highest accuracy value, whereas the detection of black chokeberries tree seedlings exhibited the lowest accuracy value. The accuracy of the detection model reached 95.10% on average (98.58% for dragon spruce tree seedlings, 91.62% for black chokeberries tree seedlings, and 95.11% for Scots pine tree seedlings). Moreover, this study has demonstrated the great significance of incorporating deep learning techniques into agriculture and forestry to aid in the statistical tasks of the number of trees, yield estimation of seedlings, and plant breeding in nurseries. Experimental results demonstrated that the YOLOv5 model could achieve accurate results and satisfactory performance for detecting and counting tree seedlings. In addition, the Yolov5 model can detect small, medium, and big objects due to the capability of its model head to achieve multi-scale prediction by generating three different sizes of feature maps. These advantages made YOLOv5 more suitable and eligible for the detection of tree seedlings.