Next Article in Journal
Habitat Distribution Pattern of Rare and Endangered Plant Magnolia wufengensis in China under Climate Change
Previous Article in Journal
Identification of Candidate Genes Involved in Bud Growth in Pinus pinaster through Knowledge Transfer from Arabidopsis thaliana Models
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Forests
Volume 14
Issue 9
10.3390/f14091766
forests-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Large-Scale and High-Accuracy Phenotyping of Populus simonii Leaves Using the Colony Counter and OpenCV

by
Sheng Zhu
1,2,*,
Heng Zhang
1,
Siyuan Chen
2,
Lei Zhang
3 and
Minren Huang
1,*
1
Key Laboratory of Forest Genetics and Biotechnology, Ministry of Education of China, Co-Innovation Center for the Sustainable Forestry in Southern China, Nanjing Forestry University, Nanjing 210037, China
2
College of Biology and the Environment, Nanjing Forestry University, Nanjing 210037, China
3
Agriculture and Rural Bureau of Pingquan City, Pingquan 067500, China
*
Authors to whom correspondence should be addressed.
Forests 2023, 14(9), 1766; https://doi.org/10.3390/f14091766
Submission received: 26 July 2023 / Revised: 23 August 2023 / Accepted: 29 August 2023 / Published: 31 August 2023
(This article belongs to the Section Forest Ecology and Management)
Browse Figures
Review Reports Versions Notes

Abstract

:
Image-based morphometric technology is broadly applicable to generate large-scale phenomic datasets in ecological, genetic and morphological studies. However, little is known about the performance of image-based measuring methods on plant morphological characters. In this study, we presented an automatic image-based workflow to obtain the accurate estimations for basic leaf characteristics (e.g., ratio of length/width, length, width, and area) from a hundred Populus simonii pictures, which were captured on Colony counter Scan1200. The image-based workflow was implemented with Python and OpenCV, and subdivided into three parts, including image pre-processing, image segmentation and object contour detection. Six image segmentation methods, including Chan-Vese, Iterative threshold, K-Mean, Mean, OSTU, and Watershed, differed in the running time, noise sensitivity and accuracy. The image-based estimates and measured values for leaf morphological traits had a strong correlation coefficient (r2 > 0.9736), and their residual errors followed a Gaussian distribution with a mean of almost zero. Iterative threshold, K-Mean, OSTU, and Watershed overperformed the other two methods in terms of efficiency and accuracy. This study highlights the high-quality and high-throughput of autonomous image-based phenotyping and offers a guiding clue for the practical use of suitable image-based technologies in biological and ecological research.

1. Introduction

Plant phenomics provides essential data for exploring the G-E-P (genotype-environment-phenotype) map, untangling the genetic architecture of complex traits, and assisting in plant breeding and cultivation management [1,2]. Image-based phenotyping has been widely used to achieve phenomic datasets, which play an increasing role in plant genetics, crop genetic improvement and their cultivation decision (e.g., cropping, fertilization, pesticide spraying, and watering) [3]. Image-based phenotyping is divided into two parts, including image capturing and image processing [4]. Imaging is well-suited for the low-cost, large-scale and rapid acquisition of high-dimensional morphological traits on plant species, particularly on their above-ground organs (e.g., leaves, stems and flowers) [5]. Some excellent tools or packages, such as OpenCV (https://opencv.org/, accessed on 12 November 2022), Scikit-image [6] and PlantCV [7], have been developed for analyzing the images of phenotypic traits.
The leaf is an important above-ground organ of herbaceous and woody plants and participates in the photosynthesis, respiration and transpiration [8]. Its morphological traits (e.g., leaf area, leaf shape, leaf length and width, as well as the length/width ratio) may provide information on the plant growth for plant geneticists and breeders [9]. These foliar morphometric characters were frequently measured in plant genetics [10]. However, large-scale phenotyping of leaf morphological traits is a time-consuming and labor-intensive task using traditional methods, such as the weighting method and grid-counting method [11]. The leaf scanner (e.g., LI-3100C Area Meter) is capable of rapidly and accurately measuring a large number of leaves, but too expensive for small research groups to afford. It is obvious that image-based phenotyping covers the shortage of these above methods. A few studies have shown that leaf morphological traits can be accurately estimated by the image-based phenotyping [12,13,14].
The factors affecting the accuracy of image-based phenotyping are of fundamental importance. The image-based estimators for leaf morphological characters may be influenced by a combination of image acquiring and processing. It remains a challenge to capture large-scale unbiased images of morphological characters using only the mono-RGB (red, green and blue) camera without auxiliary equipment to adjust and fix the position and angle of camera [15]. Image analyzing is affected directly by the quality of these acquired image datasets. Thus, it is difficult to distinguish between the factors affected by image capturing and by image processing. Nevertheless, to our knowledge, the influence of image analysis on the accuracy of estimated phenotypic values has not yet been well investigated.
In this study, to minimize the impact of image acquisition, we utilized the commercial colony counter Scan1200 to obtain the reproducible and unbiased image datasets for 100 Populus simonii leaves. We estimated the leaf morphological traits from the P. simonii images using a three-step workflow, composing of image pre-processing, image segmentation, and contour-based prediction. Leaf object was isolated from these images using four image segmentation strategies, namely threshold-based algorithm, region-based algorithm, edge-based algorithm, and K-Mean-based algorithm. The accuracy of estimates for all the image segmentation methods was assessed by comparing them with that of the leaf area meter LI-3100C.

2. Materials and Methods

2.1. Sampling and Photographing

In the research, a total of 100 leaves were harvested from two-year-old P. simonii plants grown on the ‘Beishan’ greenhouse of Nanjing Forestry University. Ten leaves were collected from each of the ten P. simonii plants grown under the same cultivation conditions. All sampled leaves were mature and fully expanded. We choose the leaves having as few diseases and insect spots as possible. In addition, the ten P. simonii plants were derived from four different genotypes, namely NX06, NX09, NX10, and JL11. NX06, NX09 and NX10 were collected from Ningxia Province, China, and JL11 was collected from Jilin Province, China.
The area of all the 100 leaves was measured using a leaf area meter LI-3100C (LI-COR Biosciences, Lincoln, NE, USA) with a resolution of 0.1 mm2. The area of each leaf was measured three times. The length and width for these leaves was directly determined with a 15 cm plastic clear ruler.
All the 100 leaves, on a clean glass petri dish, were photographed with an automatic colony counter Scan1200 (Interscience, Saint Nom, France) with a black background. The main vein of all leaves was placed on the central axis of the glass petri dish. All foliar images were saved in the BMP format. The pixel size for all the images was calculated and checked.

2.2. Data Analyzing and Visualizing

Data analysis in the study was performed on the DELL precision T5810 workstation with 16 GB RAM and an Xeon E5-1603 v3 processor. The tools/packages used for image processing, tabular data analyzing, and visualizing were run on Python (https://www.python.org/, v3.8.9, accessed on 5 August 2021). The numeric data in tabular form was handled with three Python packages, including Pandas (https://pandas.pydata.org/, v1.4.0, accessed on 8 September 2022), NumPy (https://numpy.org/, v1.19.5, accessed on 8 September 2022), and SciPy (https://www.scipy.org/, v1.6.2, accessed on 8 September 2022). The tabular data was visualized using a combination of Python’s Matplotlib (https://matplotlib.org/, v.3.4.2, accessed on 8 September 2022) and Seaborn (https://seaborn.pydata.org/, v0.11.1, accessed on 8 September 2022) packages.

2.3. Image Processing

A total of four leaf traits, including the length/width ratio, length, width, and area size, were estimated from all the P. simonii leaf images using image processing technologies. These leaf traits were the commonly used leaf morphology features and leaf functional traits in the plant ecosystem [16]. Leaf area was in direct proportion to the multiplication of leaf length and width, and the leaf area-length allometry was related to the length/width ratio [17].
The pre-processing for the P. simonii leaf images was implemented using a Python computer-vision library OpenCV (https://opencv.org/, v4.5.5, accessed on 12 November 2022). The identification of leaf regions in these P. simonii pictures was performed using seven image segmentation methods, including Chan-Vese segmentation [18], OSTU thresholding [19], the iterative thresholding [20], the adaptive thresholding [21], Canny edge detection [22], K-Mean clustering [23], and Watershed [24]. The Chan-Vese segmentation was accomplished with the chan_vese function implemented in the Python package scikit-image (https://scikit-image.org/, v0.18.2, accessed on 12 November 2022) [6]. The other six image segmentation approaches were performed using in-house Python scripts with OpenCV. The noise holes in the segmented images were closed with the ‘binary_fill_holes’ function in NumPy.

3. Results

3.1. The Morphological Traits of P. simonii Leaves

Here we adopted the area meter LI-3100C to measure three times each of all the 100 P. simonii leaves, which were derived from ten plants belonging to four genotypes or clones (e.g., JL11, NX06, NX09 and NX10). The standard deviation of the three measured area values for 99 leaves was less than 0.31, but that of the remaining one was 4.00 (Table S1). This implied that LI-3100C had the ability to generate high repeatability and unbiased measurements for total foliar area. The average measured values were used as the final leaf area values and for subsequent analysis. The leaf area for all the leaves ranged in size from 9.95 to 33.76 cm2, leaf length ranged 4.2 to 7.7 cm, leaf width ranged from 2.9 to 6.9 cm, and leaf length/width ratio from 0.89 to 2.41, respectively (Figure 1). There was a relatively high level of intra-individual or intra-genotype variability in these leaf traits.

3.2. The Overview of Image-Based Estimation of P. simonii Leaf Characteristics

In the study, we aimed to predict the basic morphological features of the P. simonii leaves from their image datasets using CV (computer vision)-based phenotyping. These predicted morphological characteristics were mainly consisting of the length/width ratio, length, width, area size, and contour of poplar leaves. The leaf area was estimated based on Grid counting or Pixel counting.
Here we present an image-based phenotyping workflow, consisting of sample collecting (Figure 2a), sample photographing (Figure 2b), and image processing (Figure 2c). The leaf image analysis was divided into three basic parts, including image pre-processing, image segmentation, and contour-based prediction (Figure 2c): (1) The original RGB photos for these P. simonii leaves (Figure 2c.0) were pre-processed via filtering the white reflected light and small noises in the images (Figure 2c.1). (2) These pre-processed RGB images were converted into grayscale images (Figure 2c.2). The grayscale images were further converted into the binary image using four segmentation technologies, including threshold-based, region-based, edge-based, and K-Mean-based strategies. The binary image only consisted of both white and black pixels, which denoted the leaf object and the background, respectively. (3) The binary images were used to find the contour and the best fitting rectangle of the foliar object (Figure 2c.3). The contour of the object in these binary images was used to infer the area size and outline of leaves. The best fitting rectangle of the object was applied to calculate the length, width, and length/width ratio of leaves.

3.3. Image Pre-Processing

The subsequent image segmentation and processing would be disturbed by a variety of fixed and random noises which were inevitability generated in image acquisition. In the research, the white aperture occurred on the leaf pictures due to the reflected light of glass culture dish on which P. simonii leaves need to be placed. Gaussian noise is a common form of random noise in these leaf images [25]. Thus, the white reflected aperture (Figure 2a) and Gaussian noises were two main types of noises to be filtered from the original images.
In order to assess the influence of the fixed and random noises on these leaf images, the grayscale images were converted from these original mono-RGB images and used for analyzing the distribution of pixel intensity values. The grayscale pixel values, ranging from 0 to 255, had the ability to show the distribution of foreground (object) and background elements in the image. There were more than two main overlapping peaks in the histogram for these original grayscale images (Figure 3a). The overlap between the adjacent peaks should lead to difficulty in the accurate segregation of object and background elements in these original leaf pictures. Therefore, the fixed and random noises in these original images might have a disruptive effect on further image segmentation.
The reflected white light and Gaussian noise were filtered from these original images. The pixel intensity histogram for these filtered images was a bimodal distribution that has two distinct peaks without an overlapping area (Figure 3b,c). It would be possible that the object and background parts can be separated precisely in the pre-processed images with distinct bimodal distribution properties.

3.4. Image Segmentation

Image segmentation is a prerequisite for the accurate measurement of leaf morphological properties in these P. simonii pictures. The main task of image segmentation is the precise separation of both object (leaf) and background parts in these pre-processed images. Here we adopted four distinct strategies, including threshold-based image segmentation, K-Means-based image segmentation, region-based image segmentation, and edge-based image segmentation, to gain the binary image that which was only composed of both leaf and background parts.

3.4.1. Threshold-Based Image Segmentation

Threshold-based image segmentation is one of the most commonly utilized strategies to easily isolate the object or foreground from background in the images. Threshold-based segmentation is roughly divided into two distinct types, including global threshold segmentation (e.g., OSTU, iterative thresholding, Mean, and Median) and local threshold segmentation (e.g., adaptive Mean and adaptive Gaussian). Global and local thresholding segmentation values are calculated on the basis of all pixel values within an entire image and a local pixel neighborhood, respectively.
Global threshold segmentation is based on the bimodal histogram composed of object and background peaks in a grayscale image. The optimal threshold is generally located at the low area (valley) between the object and background peaks. Four differential global threshold values, including OSTU, Mean, and Median, and iterative thresholding value, were used for extracting the leaf object from all the 100 P. simonii pictures. The foliar object was successfully segregated from the background in each of all the P. simonii images using three threshold-based values, including OSTU (Figure 4), Mean (Figure S1), and iterative thresholding value (Figure S2). However, there were still a handful of small noisy points in the segregated images using the three threshold values. These noisy points should not influence further analysis, such as contour detection. These Median values, ranging from 5 to 15 (Figure 5), seemed not to be suitable for the separation of the foliar object from the background in the images (Figure S3). This was probably because the Median values were far from the valley area between two peaks of gray-level bimodal histogram (Figure 3c).
The adaptive threshold is a local thresholding methodology suitable for multi-region image segmentation. The adaptive threshold may be unsuitable for segmenting the P. simonii leaf images only with a single object region. To test it, we used two adaptive threshold methods, including ADAPTIVE_THRESH_MEAN_C and ADAPTIVE_THRESH_GAUSSIAN_C (https://opencv.org/, accessed on 12 November 2022), to extract the leaf region from the entire leaf images. These P. simonii images were not clearly subdivided into both leaf and background regions using the two adaptive threshold methods (Figures S4 and S5), suggesting that the adaptive threshold could be not suitable in the case. Nevertheless, there was the distinct leaf-vein structure in the images generated using these adaptive threshold approaches. This could be very promising for analysis of leaf vein traits in plant species [26].
The Median-based image segmentation technique and two locally adaptive thresholding techniques (Mean-based and Gaussian-based) were incapable of accurately separating the foreground (leaf) region from the background region in these P. simonii pictures. Thus, the resulting images using three threshold-based segmentation techniques, including OSTU, Mean and iterative thresholding value, were used for further image processing.

3.4.2. K-Mean-Based Image Segmentation

K-Mean is one of the most widely used clustering techniques for segmenting biological and biomedical images [27]. We adopted the K-Mean method with two clusters to isolate the leaf object from the 100 P. simonii photos. Each of all the isolated P. simonii images using K-Mean clustering was composed only of both black and white parts, representing background and leaf object, respectively (Figure S6). Hence, these segmented P. simonii images were utilized for subsequent image analysis.

3.4.3. Region-Based Image Segmentation

Chan-Vese and Watershed are two frequently applied models for region-based image segmentation. Here we used the Chan-Vese and Watershed models to segment the P. simonii images. These images could be successfully divided into two regions including background and leaf parts, by using the Chan-Vese or Watershed model (Figures S7 and S8). Thus, these segmented P. simonii images with use of the Chan-Vese and Watershed models were chosen for further analysis.

3.4.4. Canny-Based Image Segmentation

The Canny-based method is a traditional edge-based segmentation which is generally used in grayscale image processing. Nevertheless, less than forty-one out of 100 P. simonii images could be accurately partitioned into leaf and background regions (Figure S9). This might be due to the fact that there was the complex and distinct leaf-vein structure in each of the P. simonii pictures. Hence, the resulting images segmented by the Canny-based method were not chosen for subsequent analysis.

3.4.5. Running Time of Six Image Segmentation Methods

Six image segmentation methods, including OSTU, Mean, iterative thresholding, K-Mean clustering, Chan-Vese and Watershed, were able to successfully separate each of all the P. simonii images into the two parts, including the foreground (leaf) and background. However, the running times of the six image segmentation approaches vary slightly (Table 1 and Figure S10). The Chan-Vese model (36.700 ± 0.847 (Mean ± σ) per image took the greatest time to segment the P. simonii images, possibly due to its maximum iteration of fifty. Slightly more than one second on average was required for the segmentation of a P. simonii picture using K-Mean clustering. It took less than 0.2 s on average to partition an image utilizing the remaining four methods, including OSTU, Mean, iterative thresholding and Watershed.

3.5. Contour-Based Predication of Morphologic Feature

3.5.1. Noises in the Segmented Images

In these previously segmented pictures utilizing the six image segmentation approaches, a substantial number of image noises appeared both inside and outside of the foreground object (P. simonii leaf). Image noises could be found along the contour line, as well as on the inner and outer edges of the leaf shapes. Their appearance could be attributed to insect gnawing holes and fungi/bacterial/virus infecting spots in P. simonii leaves, as well as the image acquisition system and image segmentation process [28]. Furthermore, the image noises were likely to interfere with the subsequent analysis [25].
The image segmentation methods might have a certain impact on the fundamental characteristics of the noises in the segmented images. We analyzed the number, area-size and locations of the noises in these grayscale images segmented using the OSTU, Mean, iterative thresholding, K-Mean clustering, Chan-Vese and Watershed (Table 2) methods. Only twenty-eight noise spots were identified in 27 out of all the 100 Watershed-segmented images. However, there are a lot of image noises in at least 99 segmented images obtained by using each of the remaining five segmentation methods. The average noise size in the segmented images produced by the Watershed method was much bigger than that produced by the other five segmentation approaches. This indicated that compared to the other five segmentation methods, the Watershed method was far more effective at eliminating small-size noise spots from the grayscale images. Additionally, several of these Mean-segmented images appeared to have image noise at the contour edges of the leaf morphologies, which could affect how accurately the basic characteristics for P. simonii leaves are estimated.
Next, we evaluated the influence of image noises on the running time and the estimated contour attribute values. The quantity of noises did not appear to significantly affect the running time of contour-based prediction for the segmented images (Pearson’s correlation coefficient r < 0.5). On the other hand, it was clear that the contour structure of the object in the segmented images might change due to these noises overlapping with their contour line. A few noises were overlaid at the contour line of the leaf object only in eight Mean-segmented images (Figure 6). The width or length of the leaf object was incorrectly estimated in the eight segmented images.

3.5.2. Contours of the Segmented Images

There might be differences in the shapes of the objects (leaves) segmented from these images using the aforementioned six methods (e.g., OSTU, Mean, Iterative thresholding, K-Mean clustering, Chan-Vese, and Watershed). The differences in the shapes segmented by these methods were then quantified using three metrics, namely Hausdorff distance [29], shape distance [30], and least absolute deviation (L1 norm) (Figure 7). The result of the three metrics showed that there were fewer differences between three paired methods, including Iterative thresholding vs. K-Mean clustering, Iterative thresholding vs. OSTU, and K-Mean clustering vs. OSTU. This indicated that the shapes of the leaves segmented by the three approaches (e.g., Iterative thresholding, K-Mean clustering, and OSTU) were relatively more similar than those of the other methods. The high shape similarity between the two global thresholding segmentation techniques, namely iterative thresholding and OSTU, can be attributed to their almost identical threshold values (Figure 5). The resemblance in shape between OSTU and K-Mean could be due to their criterion of least intra-group variance [31].

3.5.3. Morphologic Features Extracted from the Segmented Images

These binary pictures, which were segmented using the six approaches, were used to estimate the leaf morphological characteristics, including area, width, length, and ratio of length to width. The correlation coefficient (r2) between the image-based estimates and the measured values (LI-3100C) was found to be greater than 0.9995 for leaf area, >0.9885 for leaf width, >0.9854 for leaf length, and >0.9736 for leaf length-to-width ratio (Figure 8a). Additionally, the estimated values for almost all leaf lengths were slightly lower than their corresponding measured values (Figure 8a). The residual error between the image-based estimates and the measured values appeared to follow a symmetric, bell-shaped distribution (Figure 8b). To test whether these differences were derived from a Gaussian distribution, the Kolmogorov–Smirnov (KS) test was applied to investigate the cumulative distribution function (cdf) of the differences between the measured values and the values estimated using the six methods. The differences for all the six methods, except the Mean method, followed a normal distribution (p-value > 0.05).
The differences for width (KS test p-value = 0.0149) and length-to-width ratio (KS test p-value = 0.0305) traits using the Mean approach might not fit a normal distribution (p-value < 0.05). It was consistent with the result of noises and contours in the segmented images produced by the six methods. In addition, the distribution of Mean-based values was distinctly different from those of the other five methods (Figure 8b). These findings suggested that the other five methods overperformed the Mean-based method.

4. Discussion

In this study, we present an automatic image-based workflow, comprising of image pre-processing, image segmentation and object contour detection, to identify the fundamental properties of leaves in each of the 100 P. simonii pictures. The image-based workflow can be executed using in-house Python scripts which are easily coded with OpenCV-Python (https://opencv.org/, accessed on 12 November 2022) [32]. The leaf objects were extracted from these leaf pictures using six image segment approaches, including Chan-Vese, Iterative threshold, K-Mean, Mean, OSTU, and Watershed. The high correlation coefficient (r2 > 0.9736) between the image-based estimates and measured values indicated the high precision of these image-based methods for measuring leaf morphologic features. However, image noises inevitably arise during the image acquisition process, and they are essential factors influencing the accuracy of image processing [33]. In our scenario, only the Mean-based segmentation approach was sensitive to noises in a few of the denoised P. simonii images.
ImageJ is a manual or semi-automated image analysis tool applied widely in determination of morphologic traits [34]. LeafJ is a semi-automated ImageJ plugin for measuring plant leaves [35]. ImageJ has been used to measure leaf morphologic characters of P. deltoides × P. simonii [36], P. simonii [37], P. simonii × P. nigra [38], and P. tomentosa [37]. However, it is still a labor-intensive and time-consuming task to analyze hundreds or thousands of leaf pictures using only ImageJ. It is evident that our automatic OpenCV-based workflow can compensate for this deficiency.
The combination of Colony counter Scan1200 (image acquisition device) and OpenCV (image analysis package) is an effective solution for large-scale high-throughput phenotyping of above-ground organs (e.g., leaves, stems and flowers) in plant science. The method could be also used to quantificationally measure these above-ground organs during development stages or under environmental factors [39,40]. These phenotypic data are very valuable for plant biology and ecology [41]. In addition, it is clear that the method overperformed traditional phenotyping methods in terms of accuracy, efficiency, and scalability.
However, a few basic properties of the Scan1200 device leaded to some limitations. Firstly, it is difficult to acquire leaf images of plants in the field or forestland with Scan1200 having the hulking body and the power demand of 220 V AC (alternating current). Secondly, the view field of the colony counter Scan1200 is a 90 mm Petri Dish and may be too small to capture a whole leaf of some other Populus species, such as P. trichocarpa, P. deltoides and P. lasiocarpa. Clearly, the smart phone, high-definition camera and portable document scanner are portable image capturing devices, powered by the means of a USB port. Meanwhile, there are a number of emerging portable laptop chargers and portable power banks with high capacity of >80 kmAh (milliamp hours), which can power a laptop computer for nearly eight hours or more. As a result, these imaging devices have potential to cover the above-mentioned shortages of the colony counter Scan1200. Nevertheless, without their auxiliary equipment, the imaging devices are unable to capture high-quality leaf photographs with a fixed background size and without distortion [42]. Future studies will be required to seek an appropriate imaging device that is easy to carry and operate, with a relatively large view field and USB power supply, and that develops an image processing procedure suited for complex scenarios [43].

5. Conclusions

Here we presented an automatic image-based workflow to extract rapidly the accurate estimates for basic leaf characteristics from all the 100 P. simonii pictures. The image-based workflow was composed of three parts, including image pre-processing, image segmentation and object contour detection, and implemented with Python and OpenCV. Six image segmentation approaches (e.g., Chan-Vese, Iterative threshold, K-Mean, Mean, OSTU, and Watershed methods) differed in terms of the running time, noise sensitivity and accuracy. There was a high correlation coefficient (r2 > 0.9736) between the image-based estimates and measured values. Four image segmentation methods, including Iterative threshold, K-Mean, OSTU, and Watershed, overperformed the other two methods in terms of running time, noise sensitivity, and estimation accuracy. However, the image acquisition device Scan1200 had some limitations owing to its basic attributes, such as a bulky body, a 220 V power requirement, and a relatively small view-field (a circle with 45 mm radius).

Supplementary Materials

The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/f14091766/s1: Figure S1: The image segmentation of one hundred leaf figures using Mean method; Figure S2: The image segmentation of one hundred leaf figures using Iteration method; Figure S3: The image segmentation of one hundred leaf figures using Median method; Figure S4: The image segmentation of one hundred leaf figures using Adaptive mean method; Figure S5: The image segmentation of one hundred leaf figures using the Adaptive Gaussian method; Figure S6: The image segmentation of one hundred leaf figures using K-Mean method; Figure S7: The image segmentation of one hundred leaf figures using Chan-Vese method; Figure S8: The image segmentation of one hundred leaf figures using Watershed method; Figure S9: The image segmentation of one hundred leaf figures using Canny-based method; Figure S10: The relationship between the running time and noises in the segmented images. Table S1: Three repeated measurements for the total foliar area using the LI-3100C.

Author Contributions

Conceptualization and experimental design, S.Z. and M.H.; sample collection, L.Z. and S.Z.; leaf measuring and photographing, S.Z., H.Z. and S.C.; image processing and statistical analysis, S.Z. and L.Z.; writing—original draft preparation, review and editing, S.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by grants from ‘Fourteen Five-Year’ National Science and Technology Support Program (Grant No. 2021YFD2201200), and the Natural Science Foundation of Jiangsu Province (Grant No. BK20150879). The funding bodies did not have a role in the design of the study, data collection, analysis, interpretation of data, or writing the manuscript.

Data Availability Statement

All datasets presented in this study are included in the article/Supplementary figure.

Acknowledgments

We are grateful for Hui Wei and Hanmei Xu at the College of Biology and the Environment, Nanjing Forestry University, for their assistance in harvesting poplar leaves and in measuring leaf areas via LI-COR LI-3100C, respectively.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Houle, D.; Govindaraju, D.R.; Omholt, S. Phenomics: The next challenge. Nat. Rev. Genet. 2010, 11, 855–866. [Google Scholar] [CrossRef]
  2. Tao, H.; Xu, S.; Tian, Y.; Li, Z.; Ge, Y.; Zhang, J.; Wang, Y.; Zhou, G.; Deng, X.; Zhang, Z.; et al. Proximal and remote sensing in plant phenomics: 20 years of progress, challenges, and perspectives. Plant Commun. 2022, 3, 100344. [Google Scholar] [CrossRef] [PubMed]
  3. Das Choudhury, S.; Samal, A.; Awada, T. Leveraging Image Analysis for High-Throughput Plant Phenotyping. Front. Plant Sci. 2019, 10, 508. [Google Scholar] [CrossRef]
  4. Nabwire, S.; Suh, H.K.; Kim, M.S.; Baek, I.; Cho, B.K. Review: Application of Artificial Intelligence in Phenomics. Sensors 2021, 21, 4363. [Google Scholar] [CrossRef] [PubMed]
  5. Li, Z.B.; Guo, R.H.; Li, M.; Chen, Y.R.; Li, G.Y. A review of computer vision technologies for plant phenotyping. Comput. Electron. Agric. 2020, 176, 105534. [Google Scholar] [CrossRef]
  6. van der Walt, S.; Schonberger, J.L.; Nunez-Iglesias, J.; Boulogne, F.; Warner, J.D.; Yager, N.; Gouillart, E.; Yu, T.; scikit-image contributors. Scikit-image: Image processing in Python. PeerJ 2014, 2, e453. [Google Scholar] [CrossRef]
  7. Gehan, M.A.; Fahlgren, N.; Abbasi, A.; Berry, J.C.; Callen, S.T.; Chavez, L.; Doust, A.N.; Feldman, M.J.; Gilbert, K.B.; Hodge, J.G.; et al. PlantCV v2: Image analysis software for high-throughput plant phenotyping. PeerJ 2017, 5, e4088. [Google Scholar] [CrossRef]
  8. Tsukaya, H. Mechanism of leaf-shape determination. Annu. Rev. Plant Biol. 2006, 57, 477–496. [Google Scholar] [CrossRef]
  9. Fasoula, D.A.; Ioannides, I.M.; Omirou, M. Phenotyping and Plant Breeding: Overcoming the Barriers. Front. Plant Sci. 2019, 10, 1713. [Google Scholar] [CrossRef]
  10. Bhatia, N.; Runions, A.; Tsiantis, M. Leaf Shape Diversity: From Genetic Modules to Computational Models. Annu. Rev. Plant Biol. 2021, 72, 325–356. [Google Scholar] [CrossRef]
  11. Du, J.; Li, B.; Lu, X.; Yang, X.; Guo, X.; Zhao, C. Quantitative phenotyping and evaluation for lettuce leaves of multiple semantic components. Plant Methods 2022, 18, 54. [Google Scholar] [CrossRef]
  12. McDonald, S.C.; Buck, J.; Li, Z. Automated, image-based disease measurement for phenotyping resistance to soybean frogeye leaf spot. Plant Methods 2022, 18, 103. [Google Scholar] [CrossRef]
  13. Ye, M.; Zhu, X.; Gao, P.; Jiang, L.; Wu, R. Identification of Quantitative Trait Loci for Altitude Adaptation of Tree Leaf Shape with Populus szechuanica in the Qinghai-Tibetan Plateau. Front. Plant Sci. 2020, 11, 632. [Google Scholar] [CrossRef] [PubMed]
  14. Yamagishi, Y.; Kato, Y.; Ninomiya, S.; Guo, W. Image-Based Phenotyping for Non-Destructive In Situ Rice (Oryza sativa L.) Tiller Counting Using Proximal Sensing. Sensors 2022, 22, 5547. [Google Scholar] [CrossRef] [PubMed]
  15. Yang, W.; Feng, H.; Zhang, X.; Zhang, J.; Doonan, J.H.; Batchelor, W.D.; Xiong, L.; Yan, J. Crop Phenomics and High-Throughput Phenotyping: Past Decades, Current Challenges, and Future Perspectives. Mol. Plant 2020, 13, 187–214. [Google Scholar] [CrossRef] [PubMed]
  16. Schrader, J.; Shi, P.J.; Royer, D.L.; Peppe, D.J.; Gallagher, R.V.; Li, Y.R.; Wang, R.; Wright, I.J. Leaf size estimation based on leaf length, width and shape. Ann. Bot. 2021, 128, 395–406. [Google Scholar] [CrossRef]
  17. Shi, P.J.; Liu, M.D.; Ratkowsky, D.A.; Gielis, J.; Su, J.L.; Yu, X.J.; Wang, P.; Zhang, L.F.; Lin, Z.Y.; Schrader, J. Leaf area-length allometry and its implications in leaf shape evolution. Trees-Struct. Funct. 2019, 33, 1073–1085. [Google Scholar] [CrossRef]
  18. Chan, T.F.; Vese, L.A. Active contours without edges. IEEE Trans. Image Process 2001, 10, 266–277. [Google Scholar] [CrossRef]
  19. Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 1979, 9, 62–66. [Google Scholar] [CrossRef]
  20. Fornasier, M.; Rauhut, H. Iterative thresholding algorithms. Appl. Comput. Harmon A 2008, 25, 187–208. [Google Scholar] [CrossRef]
  21. Yan, F.X.; Zhang, H.; Kube, C.R. A multistage adaptive thresholding method. Pattern. Recogn Lett. 2005, 26, 1183–1191. [Google Scholar] [CrossRef]
  22. Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 1986, 8, 679–698. [Google Scholar] [CrossRef] [PubMed]
  23. Sulaiman, S.N.; Isa, N.A.M. Adaptive Fuzzy-K-means Clustering Algorithm for Image Segmentation. IEEE T Consum. Electr. 2010, 56, 2661–2668. [Google Scholar] [CrossRef]
  24. Hernandez, S.E.; Barner, K.E.; Yuan, Y. Region merging using homogeneity and edge integrity for watershed-based image segmentation. Opt. Eng. 2005, 44, 017004. [Google Scholar] [CrossRef]
  25. Tian, C.; Fei, L.; Zheng, W.; Xu, Y.; Zuo, W.; Lin, C.W. Deep learning on image denoising: An overview. Neural. Netw. 2020, 131, 251–275. [Google Scholar] [CrossRef]
  26. Price, C.A.; Munro, P.R.; Weitz, J.S. Estimates of leaf vein density are scale dependent. Plant Physiol. 2014, 164, 173–180. [Google Scholar] [CrossRef]
  27. Hedegaard, M.A.; Bergholt, M.S.; Stevens, M.M. Quantitative multi-image analysis for biomedical Raman spectroscopic imaging. J. Biophotonics 2016, 9, 542–550. [Google Scholar] [CrossRef]
  28. Li, H.; Shi, H.; Du, A.; Mao, Y.; Fan, K.; Wang, Y.; Shen, Y.; Wang, S.; Xu, X.; Tian, L.; et al. Symptom recognition of disease and insect damage based on Mask R-CNN, wavelet transform, and F-RNet. Front. Plant Sci. 2022, 13, 922797. [Google Scholar] [CrossRef]
  29. Taha, A.A.; Hanbury, A. An efficient algorithm for calculating the exact Hausdorff distance. IEEE Trans. Pattern. Anal. Mach. Intell. 2015, 37, 2153–2163. [Google Scholar] [CrossRef]
  30. Salve, S.G.; Jondhale, K.C. Shape Matching and Object Recognition Using Shape Contexts. In Proceedings of the 2010 3rd International Conference on Computer Science and Information Technology, Chengdu, China, 9–11 July 2010; pp. 471–474. [Google Scholar]
  31. Liu, D.J.; Yu, J.A. Otsu method and K-means. In Proceedings of the 2009 Ninth International Conference on Hybrid Intelligent Systems, Shenyang, China, 12–14 August 2009; Volume 1, pp. 344–349. [Google Scholar] [CrossRef]
  32. Wahab, F.; Ullah, I.; Shah, A.; Khan, R.A.; Choi, A.; Anwar, M.S. Design and implementation of real-time object detection system based on single-shoot detector and OpenCV. Front. Psychol. 2022, 13, 1039645. [Google Scholar] [CrossRef]
  33. Ilesanmi, A.E.; Ilesanmi, T.O. Methods for image denoising using convolutional neural network: A review. Complex Intell. Syst. 2021, 7, 2179–2198. [Google Scholar] [CrossRef]
  34. Rueden, C.T.; Schindelin, J.; Hiner, M.C.; DeZonia, B.E.; Walter, A.E.; Arena, E.T.; Eliceiri, K.W. ImageJ2: ImageJ for the next generation of scientific image data. BMC Bioinform. 2017, 18, 529. [Google Scholar] [CrossRef] [PubMed]
  35. Maloof, J.N.; Nozue, K.; Mumbach, M.R.; Palmer, C.M. LeafJ: An ImageJ plugin for semi-automated leaf shape measurement. J. Vis. Exp. 2013, 21, e50028. [Google Scholar] [CrossRef]
  36. Yang, W.; Yao, D.; Wu, H.; Zhao, W.; Chen, Y.; Tong, C. Multivariate genome-wide association study of leaf shape in a Populus deltoides and P. simonii F1 pedigree. PLoS ONE 2021, 16, e0259278. [Google Scholar] [CrossRef]
  37. Liu, P.; Bu, C.; Chen, P.; El-Kassaby, Y.A.; Zhang, D.; Song, Y. Enhanced genome-wide association reveals the role of YABBY11-NGATHA-LIKE1 in leaf serration development of Populus. Plant Physiol. 2023, 191, 1702–1718. [Google Scholar] [CrossRef]
  38. Zhang, Y.; Wang, B.; Qi, S.; Dong, M.; Wang, Z.; Li, Y.; Chen, S.; Li, B.; Zhang, J. Ploidy and hybridity effects on leaf size, cell size and related genes expression in triploids, diploids and their parents in Populus. Planta 2019, 249, 635–646. [Google Scholar] [CrossRef]
  39. Petrone-Mendoza, E.; Vergara-Silva, F.; Olson, M.E. Plant morpho evo-devo. Trends Plant Sci. 2023. [Google Scholar] [CrossRef]
  40. Joshi, S.; Thoday-Kennedy, E.; Daetwyler, H.D.; Hayden, M.; Spangenberg, G.; Kant, S. High-throughput phenotyping to dissect genotypic differences in safflower for drought tolerance. PLoS ONE 2021, 16, e0254908. [Google Scholar] [CrossRef]
  41. Pieruschka, R.; Schurr, U. Plant Phenotyping: Past, Present, and Future. Plant Phenomics 2019, 2019, 7507131. [Google Scholar] [CrossRef]
  42. Fahlgren, N.; Gehan, M.A.; Baxter, I. Lights, camera, action: High-throughput plant phenotyping is ready for a close-up. Curr. Opin. Plant Biol. 2015, 24, 93–99. [Google Scholar] [CrossRef]
  43. van Dijk, A.D.J.; Kootstra, G.; Kruijer, W.; de Ridder, D. Machine learning in plant science and plant breeding. iScience 2021, 24, 101890. [Google Scholar] [CrossRef] [PubMed]
Forests 14 01766 g001
Figure 1. Boxplots for leaf morphological characteristics: (a) Boxplots of the four morphological traits for ten plants, namely P00, P01, P02, P03, P04, P05, P06, P07, P08, and P09. (b) Boxplots of the four morphological traits for four clones or genotypes, including NX06, NX09, NX10, and JL11. NX06, NX09 and NX10 were collected from in Ningxia Province, China, and JL11 was collected from Jilin Province, China. The four morphological traits are the length (cm), width (cm), the ratio of length to width, and area size (cm2) of P. simonii leaves. The lower side, the inner line, and the upper side of the box in the boxplot graph are Q1 (the first quartile), median (the second quartile), and Q3 (the third quartile), respectively. The lower and upper whisker of the boxplot graph are Q1 − 1.5 × IQR (interquartile range) and Q3 + 1.5 × IQR, respectively. The values flanking the lower and upper whisker are plotted as diamond.
Figure 1. Boxplots for leaf morphological characteristics: (a) Boxplots of the four morphological traits for ten plants, namely P00, P01, P02, P03, P04, P05, P06, P07, P08, and P09. (b) Boxplots of the four morphological traits for four clones or genotypes, including NX06, NX09, NX10, and JL11. NX06, NX09 and NX10 were collected from in Ningxia Province, China, and JL11 was collected from Jilin Province, China. The four morphological traits are the length (cm), width (cm), the ratio of length to width, and area size (cm2) of P. simonii leaves. The lower side, the inner line, and the upper side of the box in the boxplot graph are Q1 (the first quartile), median (the second quartile), and Q3 (the third quartile), respectively. The lower and upper whisker of the boxplot graph are Q1 − 1.5 × IQR (interquartile range) and Q3 + 1.5 × IQR, respectively. The values flanking the lower and upper whisker are plotted as diamond.
Forests 14 01766 g001
Forests 14 01766 g002
Figure 2. The flowchart of the leaf phenotyping workflow: (a) foliar sample collecting. (b) foliar sample photographing. (c) foliar image processing. Images processing. (c.0) The original images for P. simonii leaves were captured with the Colony counter Scan1200. The white light in these images was reflected from the glass surface of the petri dish. (c.1) The pre-processing RGB images were obtained after removing the reflected light of the glass dish and filtering small noises in these pictures. (c.2) The binary images were converted from the grayscale images into which the RGB images were converted. The conversion of the grayscale images into the binary images were performed by using four different methods, including threshold-based segmentation, region-based segmentation, edge-based segmentation, and Kmean-based segmentation. (c.3) The basic features of these P. simonii leaves, including width, length, area, and contour, were predicted by means of finding their contour and fitting the best rectangle of the leaf area.
Figure 2. The flowchart of the leaf phenotyping workflow: (a) foliar sample collecting. (b) foliar sample photographing. (c) foliar image processing. Images processing. (c.0) The original images for P. simonii leaves were captured with the Colony counter Scan1200. The white light in these images was reflected from the glass surface of the petri dish. (c.1) The pre-processing RGB images were obtained after removing the reflected light of the glass dish and filtering small noises in these pictures. (c.2) The binary images were converted from the grayscale images into which the RGB images were converted. The conversion of the grayscale images into the binary images were performed by using four different methods, including threshold-based segmentation, region-based segmentation, edge-based segmentation, and Kmean-based segmentation. (c.3) The basic features of these P. simonii leaves, including width, length, area, and contour, were predicted by means of finding their contour and fitting the best rectangle of the leaf area.
Forests 14 01766 g002
Forests 14 01766 g003
Figure 3. The distribution of pixel values in the grayscale images: (a) Barplot of pixel intensity in the grayscale images converted from the original images. (b) Barplot of pixel intensity in the grayscale images after removing the white reflected light. (c) Barplot of pixel intensity in the grayscale images without the white reflected light and small background noises. The X-axis and Y-axis in these barplots were the pixel value and its corresponding proportion, respectively. The pixel values for the grayscale images ranged from 0 to 255. The error bars were colored black.
Figure 3. The distribution of pixel values in the grayscale images: (a) Barplot of pixel intensity in the grayscale images converted from the original images. (b) Barplot of pixel intensity in the grayscale images after removing the white reflected light. (c) Barplot of pixel intensity in the grayscale images without the white reflected light and small background noises. The X-axis and Y-axis in these barplots were the pixel value and its corresponding proportion, respectively. The pixel values for the grayscale images ranged from 0 to 255. The error bars were colored black.
Forests 14 01766 g003
Forests 14 01766 g004
Figure 4. The images segmented using OSTU method. This figure was formed by merging a hundred of the P. simonii images segmented with OSTU method. The ten rows of the figure showed the segmented images of ten P. simonii plants, including P00–P09 as in Figure 1, arranged from top to bottom. The ten segmented images located at each row of the figure denoted the ten leaves of a P. simonii plant (from left to right).
Figure 4. The images segmented using OSTU method. This figure was formed by merging a hundred of the P. simonii images segmented with OSTU method. The ten rows of the figure showed the segmented images of ten P. simonii plants, including P00–P09 as in Figure 1, arranged from top to bottom. The ten segmented images located at each row of the figure denoted the ten leaves of a P. simonii plant (from left to right).
Forests 14 01766 g004
Forests 14 01766 g005
Figure 5. The relationship between four diverse threshold values. These threshold values included Mean, Median, OSTU and Iterative threshold. KDE (Kernel density estimates) plots for the four threshold values were located on the diagonal of this figure and marked in green. The scatter plots were located on the lower triangles of the figure. The data points and best-fit lines in the scatter plots were indicated in blue and red, respectively.
Figure 5. The relationship between four diverse threshold values. These threshold values included Mean, Median, OSTU and Iterative threshold. KDE (Kernel density estimates) plots for the four threshold values were located on the diagonal of this figure and marked in green. The scatter plots were located on the lower triangles of the figure. The data points and best-fit lines in the scatter plots were indicated in blue and red, respectively.
Forests 14 01766 g005
Forests 14 01766 g006
Figure 6. The biased estimates of morphological features in the Mean-segmented images: (a) It indicates the overestimated length of a single leaf in a Mean-segmented images. (b–h) These indicate the overestimated width of a single leaf in seven Mean-segmented images. The bounding boxes around a leaf object in the segmented images are colored red. The contour line of the leaf object is marked in green.
Figure 6. The biased estimates of morphological features in the Mean-segmented images: (a) It indicates the overestimated length of a single leaf in a Mean-segmented images. (b–h) These indicate the overestimated width of a single leaf in seven Mean-segmented images. The bounding boxes around a leaf object in the segmented images are colored red. The contour line of the leaf object is marked in green.
Forests 14 01766 g006
Forests 14 01766 g007
Figure 7. The pairing similarity between the objects isolated from these poplar leaf images using six different methods: (a–c) show the Hausdorff distance, the shape distance and the absolute difference between the leaf objects segmented from an image with the six methods, including Chan-Vese, Iterative threshold, Kmean, Mean, OSTU and Watershed. The X-axis represents the two methods which need to be compared in pairs, such as ChanVese vs. Iterative and ChanVese vs. Kmean. The lower and upper whisker of the boxplot graph are Q1 − 1.5 × IQR (interquartile range) and Q3 + 1.5 × IQR, respectively. The values flanking the lower and upper whisker are plotted as diamond.
Figure 7. The pairing similarity between the objects isolated from these poplar leaf images using six different methods: (a–c) show the Hausdorff distance, the shape distance and the absolute difference between the leaf objects segmented from an image with the six methods, including Chan-Vese, Iterative threshold, Kmean, Mean, OSTU and Watershed. The X-axis represents the two methods which need to be compared in pairs, such as ChanVese vs. Iterative and ChanVese vs. Kmean. The lower and upper whisker of the boxplot graph are Q1 − 1.5 × IQR (interquartile range) and Q3 + 1.5 × IQR, respectively. The values flanking the lower and upper whisker are plotted as diamond.
Forests 14 01766 g007
Forests 14 01766 g008
Figure 8. The relationship between measured values and image-based estimates: (a) shows the correlation between measured values and image-based estimates for four foliaceous traits, including leaf area, length, width and length-to-width ratio. (b) indicates the density distribution of the difference between measured values and image-based estimates for the four traits. Chan-Vese, Iterative threshold, K-Mean, Mean, OSTU and Watershed were colored blue, yellow, green, red, violet and brown, respectively.
Figure 8. The relationship between measured values and image-based estimates: (a) shows the correlation between measured values and image-based estimates for four foliaceous traits, including leaf area, length, width and length-to-width ratio. (b) indicates the density distribution of the difference between measured values and image-based estimates for the four traits. Chan-Vese, Iterative threshold, K-Mean, Mean, OSTU and Watershed were colored blue, yellow, green, red, violet and brown, respectively.
Forests 14 01766 g008
Table 1. The running time (seconds) of image segmentation.
Table 1. The running time (seconds) of image segmentation.
MethodMeanStdMinimumMaximumMedian
Mean0.0090.0050.0000.0240.008
Iterative Threshold0.1510.0080.1370.1980.151
OSTU0.1800.0640.1210.3830.154
K-Mean1.1140.1510.7711.4631.105
Watershed0.0660.0250.0500.3100.062
Chan-Vese36.7000.84736.31143.07136.458
[Note] Std: standard deviation (σ).
Table 2. The noises in the segmented images.
Table 2. The noises in the segmented images.
MethodTotal ImageNoise PointsNoise/ImageArea/ImageArea/Noise
Chan-Vese993143.17154.0748.58
Iteration1003943.9478.2019.85
K-Mean1008008.00371.9046.49
Mean100947094.701798.4718.99
OSTU1003993.9978.7019.72
WaterShed27281.04134.85130.04
[Note] Area size was indicated by the total number of pixels in image noises.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Zhu, S.; Zhang, H.; Chen, S.; Zhang, L.; Huang, M. Large-Scale and High-Accuracy Phenotyping of Populus simonii Leaves Using the Colony Counter and OpenCV. Forests 2023, 14, 1766. https://doi.org/10.3390/f14091766

AMA Style

Zhu S, Zhang H, Chen S, Zhang L, Huang M. Large-Scale and High-Accuracy Phenotyping of Populus simonii Leaves Using the Colony Counter and OpenCV. Forests. 2023; 14(9):1766. https://doi.org/10.3390/f14091766

Chicago/Turabian Style

Zhu, Sheng, Heng Zhang, Siyuan Chen, Lei Zhang, and Minren Huang. 2023. "Large-Scale and High-Accuracy Phenotyping of Populus simonii Leaves Using the Colony Counter and OpenCV" Forests 14, no. 9: 1766. https://doi.org/10.3390/f14091766

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop