# Tassel-YOLO: A New High-Precision and Real-Time Method for Maize Tassel Detection and Counting Based on UAV Aerial Images

^{1}

^{2}

^{3}

^{4}

^{*}

^{†}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

#### 2.1. YOLO Model

#### 2.2. Global Attention Mechanism

#### 2.3. Gsconv

_{1}. Deep depthwise separable convolution is applied to half of the channels, while regular convolution is applied to the other half. The outputs of both convolutions are concatenated for feature fusion. Subsequently, the information generated by SC is permeated through shuffle to various parts of the information generated by DSC. Finally, the output channel number in the feature map is C

_{2}. The mathematical expression of the GSconv module is given by Equation (1).

## 3. Methods

#### 3.1. Tassel-YOLO Model Architecture

#### 3.2. Siou Loss Function

#### 3.2.1. Angle Cost

#### 3.2.2. Distance Cost

#### 3.2.3. Shape Cost

#### 3.2.4. IoU Cost

#### 3.2.5. SIoU Cost

## 4. Experimental Material

#### 4.1. The Establishment of the Dataset

#### 4.2. Data Augmentation

_{x1}is a batch of samples and batch

_{y1}is the corresponding labels, batch

_{x2}is another batch of samples and batch

_{y2}is the corresponding labels. λ is the mixing parameter calculated from the Beta distribution with parameters α and β, and the principal formula of Mixup is obtained accordingly.

_{x}refers to the mixed batch samples, and mixed_batch

_{y}refers to the corresponding labels. Mixup data augmentation increases the diversity of the training set by performing linear interpolation between different images and labels to generate new training data.

## 5. Experiment Results

#### 5.1. Experimental Platform and Evaluation Indicators

#### 5.2. Training Comparison with Other Models

#### 5.3. Counting and Detection Results

#### 5.4. Contrast Experiment Results of Introducing Attention Mechanism

#### 5.5. Ablation Experiment

## 6. Conclusions and Future Work

- This study focuses on the research and development of real-time detection tasks for maize tassels. In the future, as more data become available for various plant species and quantities, we will continue to optimize Tassel-YOLO and apply our model to broader fields, such as wheatear detection and ears of millet detection.
- Hyperspectral images can provide richer spectral information, and using hyperspectral images for tassel detection can provide more comprehensive and accurate data support. This is also a future research direction worth exploring.
- During the growth process of maize, which includes multiple growth stages, this study only investigated the detection and counting of the tasseling stage. In the future, we will experimentally analyze images from other growth stages to obtain a more comprehensive assessment of maize quantity.
- This study achieved the counting of tassels at a local position of a field represented by a single image. However, calculating the tassel count of the entire maize field through image overlap also has certain research significance.

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- U.S. Department of Agriculture. Website of Foreign Agriculture Service. Available online: https://ipad.fas.usda.gov/cropexplorer/cropview/commodityView.aspx?cropid=0440000 (accessed on 10 April 2023).
- Wang, B.; Yang, G.; Yang, H.; Gu, J.; Xu, S.; Zhao, D.; Xu, B. Multiscale Maize Tassel Identification Based on Improved RetinaNet Model and UAV Images. Remote Sens.
**2023**, 15, 2530. [Google Scholar] [CrossRef] - Chen, X.; Pu, H.; He, Y.; Lai, M.; Zhang, D.; Chen, J.; Pu, H. An Efficient Method for Monitoring Birds Based on Object Detection and Multi-Object Tracking Networks. Animals
**2023**, 13, 1713. [Google Scholar] [CrossRef] [PubMed] - Zhao, L.; Zhu, M. MS-YOLOv7:YOLOv7 Based on Multi-Scale for Object Detection on UAV Aerial Photography. Drones
**2023**, 7, 188. [Google Scholar] [CrossRef] - Zou, H.; Lu, H.; Li, Y.; Liu, L.; Cao, Z. Maize tassels detection: A benchmark of the state of the art. Plant Methods
**2020**, 16, 108. [Google Scholar] [CrossRef] [PubMed] - Liu, Y.; Cen, C.; Che, Y.; Ke, R.; Ma, Y.; Ma, Y. Detection of Maize Tassels from UAV RGB Imagery with Faster R-CNN. Remote Sens.
**2020**, 12, 338. [Google Scholar] [CrossRef] [Green Version] - Ji, M.; Yang, Y.; Zheng, Y.; Zhu, Q.; Huang, M.; Guo, Y. In-field automatic detection of maize tassels using computer vision. Inf. Process. Agric.
**2021**, 8, 87–95. [Google Scholar] [CrossRef] - Mirnezami, S.V.; Srinivasan, S.; Zhou, Y.; Schnable, P.S.; Ganapathysubramanian, B. Detection of the Progression of Anthesis in Field-Grown Maize Tassels: A Case Study. Plant Phenomics
**2021**, 2021, 4238701. [Google Scholar] [CrossRef] - Falahat, S.; Karami, A. Maize tassel detection and counting using a YOLOv5-based model. Multimed. Tools Appl.
**2022**, 82, 19521–19538. [Google Scholar] [CrossRef] - Liu, Y.; Shao, Z.; Hoffmann, N. Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions. arXiv
**2021**, arXiv:2112.05561. [Google Scholar] - Gevorgyan, Z. SIoU Loss: More Powerful Learning for Bounding Box Regression. arXiv
**2022**, arXiv:2205.12740. [Google Scholar] - Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
- Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv
**2022**, arXiv:2207.02696. [Google Scholar] [CrossRef] - Mnih, V.; Heess, N.; Graves, A.; Kavukcuoglu, K. Recurrent Models of Visual Attention. arXiv
**2014**, arXiv:1406.6247. [Google Scholar] - Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 11534–11542. [Google Scholar]
- Wang, H.; Fan, Y.; Wang, Z.; Jiao, L.; Schiele, B. Parameter-Free Spatial Attention Network for Person Re-Identification. arXiv
**2018**, arXiv:1811.12150. [Google Scholar] - Hu, J.; Shen, L.; Albanie, S.; Sun, G.; Wu, E. Squeeze-and-Excitation Networks. arXiv
**2019**, arXiv:1709.01507. [Google Scholar] - Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv
**2017**, arXiv:1704.04861. [Google Scholar] - Li, H.; Li, J.; Wei, H.; Liu, Z.; Zhan, Z.; Ren, Q. Slim-neck by GSConv: A better design paradigm of detector architectures for autonomous vehicles. arXiv
**2022**, arXiv:2206.02424. [Google Scholar] - Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv
**2020**, arXiv:2004.10934. [Google Scholar] - Liu, W.; Quijano, K.; Crawford, M.M. YOLOv5-Tassel: Detecting Tassels in RGB UAV Imagery With Improved YOLOv5 Based on Transfer Learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2022**, 15, 8085–8094. [Google Scholar] [CrossRef] - Ye, M.; Wang, H.; Xiao, H. Light-YOLOv5: A Lightweight Algorithm for Improved YOLOv5 in PCB Defect Detection. In Proceedings of the 2023 IEEE 2nd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), Changchun, China, 24–26 February 2023; pp. 523–528. [Google Scholar] [CrossRef]
- Du, S.; Zhang, B.; Zhang, P.; Xiang, P. An Improved Bounding Box Regression Loss Function Based on CIOU Loss for Multi-scale Object Detection. In Proceedings of the 2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML), Chengdu, China, 16–18 July 2021; pp. 92–98. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, H.; Xin, Z. Efficient Detection Model of Steel Strip Surface Defects Based on YOLO-V7. IEEE Access
**2022**, 10, 133936–133944. [Google Scholar] [CrossRef] - Tzutalin, D.L. Git Code. 2015. Available online: https://github.com/tzutalin/labelImg (accessed on 10 April 2023).
- Kumar, T.; Turab, M.; Raj, K.; Mileo, A.; Brennan, R.; Bendechache, M. Advanced Data Augmentation Approaches: A Comprehensive Survey and Future directions. arXiv
**2023**, arXiv:2301.02830. [Google Scholar] - Jiang, W.; Zhang, K.; Wang, N.; Yu, M. MeshCut data augmentation for deep learning in computer vision. PLoS ONE
**2020**, 15, e0243613. [Google Scholar] [CrossRef] [PubMed] - Zhang, H.; Cisse, M.; Dauphin, Y.N.; Lopez-Paz, D. mixup: Beyond Empirical Risk Minimization. arXiv
**2018**, arXiv:1710.09412. [Google Scholar] - Robbins, H.; Monro, S. A Stochastic Approximation Method. Ann. Math. Stat.
**1951**, 22, 400–407. [Google Scholar] [CrossRef] - Cao, L.; Xiao, Z.; Liao, X.; Yao, Y.; Wu, K.; Mu, J.; Pu, H. Automated Chicken Counting in Surveillance Camera Environments Based on the Point Supervision Algorithm: LC-DenseFCN. Agriculture
**2021**, 11, 493. [Google Scholar] [CrossRef] - Foss, T.; Myrtveit, I.; Stensrud, E. MRE and heteroscedasticity: An empirical validation of the assumption of homoscedasticity of the magnitude of relative error. In Proceedings of the ESCOM, 12th European Software Control And Metrics Conference, Maastricht, The Netherlands, 2–4 April 2001; pp. 157–164. [Google Scholar]
- Terven, J.; Cordova-Esparza, D. A Comprehensive Review of YOLO: From YOLOv1 to YOLOv8 and Beyond. arXiv
**2023**, arXiv:2304.00501. [Google Scholar] - Kumar, A.; Taparia, M.; Rajalakshmi, P.; Guo, W.; Naik, B.; Marathi, B.; Desai, U.B. UAV Based Remote Sensing for Tassel Detection and Growth Stage Estimation of Maize Crop Using Multispectral Images. In Proceedings of the IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA, 26 September–2 October 2020; pp. 1588–1591. [Google Scholar] [CrossRef]

**Figure 10.**Some effects of data augmentation methods. (

**a**) Original image; (

**b**) Rotation; (

**c**) Equal scaling; (

**d**) Color dithering; (

**e**) Mosaic; (

**f**) Mix up.

**Figure 12.**The results of maize tassel detection by Tassel-YOLO. (

**a**) The detection performance of maize tassels at different sizes; (

**b**) Overall image detection performance.

Date | Weather | Device | Resolution | FPS | Image Sensor |
---|---|---|---|---|---|

16 June 2022 | Sunny | DJI Mavic drone | 12 MP | 24@1080P | 1-inch CMOS |

2 July 2022 | Sunny | DJI Mavic drone | 12 MP | 24@1080P | 1-inch CMOS |

Model | mAP@0.5 | Precision | Recall | F1 | FPS |
---|---|---|---|---|---|

YOLOv4 | 89.10% | 88.01% | 85.92% | 86.95% | 55 |

YOLOv5 | 93.42% | 91.23% | 89.13% | 90.17% | 86 |

YOLOv7 | 94.71% | 92.32% | 91.74% | 92.03% | 69 |

YOLOv8 | 94.26% | 92.14% | 92.92% | 92.53% | 75 |

Tassel-YOLO | 96.14% | 93.16% | 93.21% | 93.18% | 74 |

Tassel-YOLO | YOLOv8 | YOLOv7 | YOLOv5 | YOLOv4 | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Group | NMC | NAC | CA | MRE (%) | NAC | CA | MRE (%) | NAC | CA | MRE (%) | NAC | CA | MRE (%) | NAC | CA | MRE (%) |

1 | 380 | 368 | 96.8% | 0.32 | 359 | 94.5% | 0.55 | 364 | 95.8% | 0.42 | 358 | 94.2% | 0.58 | 347 | 91.3% | 0.87 |

2 | 791 | 771 | 97.5% | 0.25 | 756 | 95.6% | 0.44 | 763 | 96.5% | 0.35 | 754 | 95.3% | 0.47 | 729 | 92.2% | 0.78 |

3 | 1248 | 1221 | 97.8% | 0.22 | 1211 | 97.0% | 0.30 | 1209 | 96.9% | 0.31 | 1193 | 95.6% | 0.44 | 1158 | 92.8% | 0.72 |

4 | 1682 | 1650 | 98.1% | 0.19 | 1639 | 97.4% | 0.26 | 1633 | 97.1% | 0.29 | 1615 | 96.0% | 0.40 | 1569 | 93.3% | 0.67 |

Attention Mechanism | Precision | Recall | F1 | mAP@0.5 | FLOPs | Parameters | ||
---|---|---|---|---|---|---|---|---|

SE | CBAM | GAM | ||||||

× | × | × | 92.32% | 91.74% | 92.03% | 94.71% | 103.2 G | 36.48 M |

√ | × | × | 92.92% | 89.48% | 91.17% | 94.33% | 103.3 G | 36.62 M |

× | √ | × | 93.57% | 91.24% | 92.39% | 94.83% | 103.9 G | 37.63 M |

× | × | √ | 92.84% | 92.86% | 92.85% | 95.84% | 111.5 G | 43.98 M |

Methods | mAP@0.5 | F1 | FLOPs | Parameters | Inference Time (ms) |
---|---|---|---|---|---|

YOLOv7 | 94.71% | 92.03% | 103.2 G | 36.48 M | 14.5 |

YOLOv7 + GAM | 95.84% | 92.85% | 111.5 G | 43.98 M | 15.6 |

YOLOv7 + Slim Neck | 95.21% | 91.87% | 82.9 G | 26.69 M | 12.3 |

YOLOv7 + SIoU | 94.92% | 92.16% | 103.2 G | 36.48 M | 14.5 |

Tassel-YOLO | 96.14% | 93.18% | 91.8 G | 32.37 M | 13.5 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pu, H.; Chen, X.; Yang, Y.; Tang, R.; Luo, J.; Wang, Y.; Mu, J.
Tassel-YOLO: A New High-Precision and Real-Time Method for Maize Tassel Detection and Counting Based on UAV Aerial Images. *Drones* **2023**, *7*, 492.
https://doi.org/10.3390/drones7080492

**AMA Style**

Pu H, Chen X, Yang Y, Tang R, Luo J, Wang Y, Mu J.
Tassel-YOLO: A New High-Precision and Real-Time Method for Maize Tassel Detection and Counting Based on UAV Aerial Images. *Drones*. 2023; 7(8):492.
https://doi.org/10.3390/drones7080492

**Chicago/Turabian Style**

Pu, Hongli, Xian Chen, Yiyu Yang, Rong Tang, Jinwen Luo, Yuchao Wang, and Jiong Mu.
2023. "Tassel-YOLO: A New High-Precision and Real-Time Method for Maize Tassel Detection and Counting Based on UAV Aerial Images" *Drones* 7, no. 8: 492.
https://doi.org/10.3390/drones7080492