Quality Assessment of Dried White Mulberry (Morus alba L.) Using Machine Vision

Abstract

Over the past decade, the fresh white mulberry (Morus alba L.) fruit has gained growing interest due to its superior health and nutritional characteristics. While white mulberry is consumed as fresh fruit in several countries, it is also popular in dried form as a healthy snack food. One of the main challenges that have prevented a wider consumer uptake of this nutritious fruit is the non-uniformity in its quality grading. Therefore, identifying a reliable quality grading tool can greatly benefit the relevant stakeholders. The present research addresses this need by developing a novel machine vision system that combines the key strengths of image processing and artificial intelligence. Two grades (i.e., high- and low-quality) of white mulberry were imaged using a digital camera and 285 colour and textural features were extracted from their RGB images. Using the quadratic sequential feature selection method, a subset of 23 optimum features was identified to classify samples into two grades using artificial neural networks (ANN) and support vector machine (SVM) classifiers. The developed system under both classifiers achieved the highest correct classification rate (CCR) of 100%. Indeed, the latter approach offered a smaller mean squared error for the training and test sets. The developed model’s high accuracy confirms the machine vision’s suitability as a reliable, low-cost, rapid, and intelligent tool for quality monitoring of dried white mulberry.

1. Introduction

With an increasing demand for healthy and nutritious agri-food products, berries have gained a lot of interest around the globe. For instance, the global import value of fresh mulberries, raspberries, loganberries, and blackberries has risen from USD 1.7 billion in 2014 to USD 3.8 billion in 2021. The top five importers of berries above include the United States, Germany, Canada, United Kingdom, and Spain, with 43%, 10%, 9%, 9%, and 6% of the global market import, respectively [1].

Among berry fruits, white mulberry is of great interest as it contains carbohydrates, protein, fiber, fat, vitamins and minerals [2,3]. Moreover, the phenolic compounds of white mulberry have a wide range of antioxidant and antimutagenic activities and anti-cancer properties [3,4,5].

White mulberry can be consumed both in fresh or dried form (as a healthy snack food) [4]. However, the main challenge the industry is facing in trading this nutritious fruit, specifically in its dried form, is the inconsistency in its quality grading due to a lack of well-developed tools. The quality of the fresh fruit and the drying conditions are the main factors that determine the dried white mulberry quality. High-quality products usually have negligible damaged/broken components and are milky in colour. In contrast, lower-quality dried samples have several damaged/broken/smashed pieces and are darker in colour. The industry’s common approach for grading dried mulberry is a visual inspection conducted by skilled personnel. However, manual assessment is time-consuming, expensive, and a subjective task. Therefore, developing a rapid and intelligent grading tool can benefit the industry and provide consistent quality to consumers.

Quality monitoring is an important step in production to achieve high-marketability products with optimum use of resources [6]. Over the past decade, the potential of optical and machine vision techniques for agri-food products’ quality monitoring has remarkably progressed [7,8,9,10,11,12,13,14]. For instance, machine vision systems have been utilized for the evaluation of agri-food products’ maturity levels [15,16], mechanical damage [17,18,19], soundness [20,21,22], growth parameters [18,23], and grading [24,25,26].

The most frequently used machine vision tools have been imaging- or spectroscopic-based systems [27,28]. The former offers lower costs, while the latter has the capability to explore a sample’s chemical constituents. Indeed, the majority of developed models have been sample- and application-specific due to the nature of the products. So, each product and/or application needs its own calibration and algorithm tools [16].

Scholars have demonstrated the application of imaging-based machine vision tools for the maturity evaluation of various products, including fresh white mulberry [29]. However, despite several agri-food products being extensively researched for non-destructive quality determination, our thorough literature review indicates that there has been no attempt to explore machine vision for the quality evaluation of dried mulberry. The present work aims to fill this knowledge gap by developing a novel machine vision system that combines the key strengths of image processing and artificial intelligence. The authors believe the outcome of the present study can open new horizons toward automating mulberry quality monitoring.

2. Materials and Methods

2.1. Samples

The dried white mulberry (Morus alba L.) (Figure 1a) product was purchased from a local market in Marivan City, Kurdistan, Iran. A panel of experts manually graded the samples into two grades (i.e., high- and low-quality) (Figure 1b) based on the appearance of the product. A total of 100 representative samples were selected for this study (50 samples for each grade).

2.2. Image Acquisition

The individual samples were placed on white paper to be imaged (see Figure 1c) using a digital camera (model J7, Samsung Corp., Seoul, Korea). The camera’s resolution was 13 MP.

2.3. Image Processing

The image processing steps involved preprocessing, feature extraction and feature analysis. Figure 2 depicts the proposed algorithm’s different steps, which will be discussed in further detail later. The image analysis was performed in MATLAB (Version 2016a, Mathworks Inc., Waltham, MA, USA).

2.3.1. Image Preprocessing

To separate the object’s image from the background, the acquired RGB (Red Green Blue) images were processed using Equation (1) (empirical equation):

$$BI=R-0.5G-0.5B$$

(1)

where BI is a monocolour image [30,31], and R, G, and B are the channels of the sample image in RGB colour space, respectively. The obtained BI images were converted to binary (black and white) images to form binary masks. Erosion and dilation operations were employed for noise removal. The initial RGB images were multiplied by the binary masks to select regions of interest (ROI).

2.3.2. Feature Extraction

The successful use of various colour and texture features in agri-food quality evaluation has been previously demonstrated [29,32]. Herein, a similar approach to [16] was utilized. First, the images in RGB colour space were transferred to six other colour spaces, including L*a*b*, HSV, NRGB, CrCgCb, I1I2I3, and gray level spaces [25,30,33,34]. Detailed information about the provided colour spaces and the corresponding individual channels are available elsewhere [30,35,36,37]. The data of utilized colour spaces were recorded from 19 channels, including R, G, B, L*, a*, b*, H, S, V, nr, ng, nb, cr, cg, cb, I1, I2, I3, and gray level [16].

From the colour spaces mentioned above, 10 colour features were calculated, namely maximum, mean, minimum, mode, median, standard deviation, coefficient of variation, kurtosis, skewness, and covariance [15,31,35,38]. Moreover, from the gray-level co-occurrence matrix (GLCM) of ROI images, 5 texture features were calculated, namely energy, entropy, correlation, homogeneity and contrast [25,30,34]. Overall, for each sample, 285 features were recorded (15 features × 19 channels).

2.3.3. Feature Analysis

2.3.3.1. Feature Selection

It is well-established that when dealing with large data sets, feature selection could reduce computational time and power while enhancing efficiency. This work employed a quadratic-based sequential feature selection method according to [16,30,31,34,39] to identify optimum features among the available 285 features (see Section 2.3.2).

2.3.3.2. Feature Classification

Shallow artificial neural networks (ANNs) [40,41,42,43] and support vector machine (SVM) methods [38] were used to classify the fruit into two classes: high- and low-quality.

The ANN structure consisted of three (viz. input, hidden and target) layers. The number of optimum features (Section 2.3.3.1) and desired classes (i.e., high- and low-quality mulberry) determined the number of neurons in the input and target layer, respectively. The number of neurons in the hidden layer varied from 2 to 20 to identify the optimum structure. The activation functions for the hidden and target layers were tangent sigmoid and pure line, respectively [16]. The data were randomly divided into three groups: 60% for training, 20% for validation, and 20% for testing. Two main measures were evaluated to assess the performance of various ANN-based classifiers: the correlation coefficient (R) and correct classification rate (CCR).

In the case of the SVM classifier, the data were divided into two groups: 75% for training and 25% for testing. The results of the SVM method were compared with that of ANN in terms of CCR and mean squared error (MSE).

3. Results

3.1. Feature Set Optimization

Using the feature selection algorithm (Section 2.3.3.1), 23 out of 285 features were identified as optimum. As seen in

Table 1. The optimum features of dried white mulberry. Grades 1 and 2 represent high- and low-quality samples, respectively.

No	Feature	Channel	Mean
			Grade 1	Grade 2
1	Mean	Gray	0.48	0.35
2	Median	Gray	0.48	0.34
3	Mean	R	0.52	0.38
4	Coefficient of variation	R	0.08	0.14
5	Median	R	0.52	0.38
6	Mean	G	0.47	0.33
7	Median	G	0.47	0.33
8	Mean	B	0.42	0.31
9	Kurtosis	B	3.68	5.58
10	Mode	B	0.417	0.30
11	Mean	L*	74.44	65.19
12	Standard deviation	L*	2.80	3.82
13	Median	L*	74.75	64.88
14	Skewness	L*	3.23	4.85
15	Coefficient of variation	L*	8.07	15.00
16	Mode	I1	0.48	0.32
17	Entropy	I3	1.30	0.19
18	Median	H	0.09	0.07
19	Mean	V	0.52	0.38
20	Standard deviation	V	0.04	0.05
21	Coefficient of variation	V	0.08	0.14
22	Median	V	0.52	0.38
23	Mode	V	0.53	0.37

, the mean values of the optimum features were substantially different for high- and low-quality mulberry. These optimum features were used as the inputs of the ANN classifiers.

3.2. Feature Classification

Table 2. The developed ANN classification structures with their characteristics.

No.	Structure	Training Data		Validation Data		Testing Data		Total Data
		R *	CCR **	R	CCR	R	CCR	R	CCR
1	23-3-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
2	23-4-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
3	23-5-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
4	23-6-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
5	23-7-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
6	23-8-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
7	23-9-2	1.00	100.00	1.00	100.00	0.90	95.00	0.98	99.00
8	23-10-2	1.00	100.00	1.00	100.00	0.92	95.00	0.98	99.00
9	23-11-2	1.00	100.00	0.99	100.00	0.69	95.00	0.93	99.00
10	23-12-2	1.00	100.00	1.00	100.00	0.74	95.00	0.94	99.00
11	23-13-2	1.00	100.00	1.00	100.00	0.86	95.00	0.97	99.00
12	23-14-2	1.00	100.00	1.00	100.00	0.99	100.00	1.00	100
13	23-15-2	1.00	100.00	1.00	100.00	0.87	95.00	0.97	99.00
14	23-16-2	1.00	100.00	1.00	100.00	0.81	100.00	0.96	100.00
15	23-17-2	1.00	100.00	1.00	100.00	0.87	95.00	0.97	99.00
16	23-18-2	1.00	100.00	0.92	90.00	0.66	95.00	0.90	97.00
17	23-19-2	1.00	100.00	1.00	100.00	0.84	95.00	0.97	99.00
18	23-20-2	1.00	100.00	0.99	100.00	0.86	95.00	0.97	99.00

* R represents the correlation coefficient. ** CCR represents the correct classification rate (%).

shows the performance of different ANN structures discussed in Section 2.3.3.2. The best performance was achieved under the 23-14-2 structure (the three numbers represent the neurons in the input, hidden, and output layers, respectively). The CCR for training, validation, testing and entire data set under the 23-14-2 structure was 100%. The correlation coefficient for the training, validation and the whole data set was 1.

The confusion matrices of the optimum model for training, validation, testing, and the whole data set are presented in

Table 3. The confusion matrix of the optimum classifier (ANN with 23-14-2 structure). Grades 1 and 2 represent the number of high- and low-quality samples, respectively.

Data Set	Actual	Predicted		CCR
		Grade 1	Grade 2
Training	Grade 1	30	0	100%
	Grade 2	0	30
Validation	Grade 1	10	0	100%
	Grade 2	0	10
Testing	Grade 1	10	0	100%
	Grade 2	0	10
Total	Grade 1	50	0	100%
	Grade 2	0	50

. The optimum classifier can successfully discriminate different grades of the dried white mulberry in all data sets.

Figure 3 shows the MSE of the optimum classifier model in the validation step. This figure gave the minimum value of the mean squared error as 0.00066 after six epochs. The MSEs for training and test set at the same epoch were around 0.000005 and 0.01, respectively. Figure 4 illustrates the regression lines of the optimum classifier model for training, validation, testing, and the whole data set. The correlation coefficients of the classifier for all datasets were in excess of 0.996, except for the test data set (0.985).

The results of the SVM classifier for training, testing, and the whole data set have been presented in

Table 4. The confusion matrix of the SVM classifier. Grades 1 and 2 represent the number of high- and low-quality samples, respectively.

Data Set	Actual	Predicted		CCR
		Grade 1	Grade 2
Training	Grade 1	38	0	100%
	Grade 2	0	37
Testing	Grade 1	12	0	100%
	Grade 2	0	13
Total	Grade 1	50	0	100%
	Grade 2	0	50

. It is apparent that the two grades can be distinguished by 100 % accuracy. The MSE values of the classification model for training, testing, and the whole data set were 0.013, 0.040, and 0.01, respectively.

4. Discussion

Our results confirm the imaging-based machine vision tool’s capability for rapid quality monitoring of dried white mulberry. Both SVM and ANN classifiers provided 100% accuracy in distinguishing the high- and low-quality dried mulberry. Indeed, the latter approach can be considered superior as it offers lower MSE.

To the best of our knowledge, the present work is the first effort at quality monitoring of dried white mulberry using a low-cost machine vision system. The obtained results are in reasonable agreement with similar maturity classifiers reported by Kheiralipour et al. [16] and Salam et al. [31]. We believe the proposed system is useful for improving product marketability by sending high-quality products with uniform colour and texture to the target markets. Moreover, the system is beneficial from a waste management perspective by sending the lower-quality products to special processing units, which can be used as input for different products, such as pet foods. Hence, this research can be very useful in enhancing the postharvest quality determination and processing of the white mulberry fruit.

To improve the reliability of the developed model, future works need to evaluate the performance of the proposed intelligent algorithms on larger datasets. Acquiring more datasets may be achieved by imaging more samples or through implementing data augmentation tools such as Generative Adversarial Networks (GANs) [44].

To facilitate the uptake of the developed machine vision tool by farmers and industry stakeholders, the system should segregate the samples according to their quality parameters in a rapid and real-time manner. One strategy to reach such a goal is to integrate the imaging tools into an intelligent conveyer belt-based classifier system, similar to the one proposed by Azarmdel et al. [29]. Hence, another potential topic for future research could be to evaluate the performance of the proposed machine vision tools on such an opto-electro-mechanical system [29]. Once a reliable performance is achieved, the system can be integrated into the mulberry drying industry for quality control and optimization.

One should note that the capital costs associated with the proposed quality assessment system may be the main prohibiting factor for the farmers and industry to uptake the mentioned technology. However, the return on such extra capital cost, especially for commercial installations will be faster processing, improved classification accuracy, and the need for fewer human personnel. Thus, it can be considered a safe, short-run return and reasonable investment, which may even qualify for government subsidies in some jurisdictions.

Ultimately, one should consider that while the developed image processing algorithm in the present research are sample-specific, they can be modified to apply to other similar fruit such as figs (Ficus carica) and other berries. Moreover, upon further modifications and optimization, the proposed model may be able to classify fruits according to other quality characteristics such as soundness and maturity levels.

5. Conclusions

A rapid, low-cost, non-destructive machine vision-based approach was developed to assess the quality of dried white mulberry samples. Using SVM and ANN classifiers, the samples were classified into two grades (i.e., high- and low-quality) with 100% correct classification accuracy. The mean squared error of the test set for the aforementioned classifiers were 0.04 and 0.01, respectively, indicating the better performance of the ANN classifier.

Considering the promising performance of the proposed machine vision system, the future effort should focus on adopting the proposed technology to develop an opto-electro-mechanical system that can operate in real-time and at an industrial scale. Upon modification and optimization, such systems will enable rapid and automated grading of mulberry (or similar fruits) based on various characteristics such as appearance, soundness and maturity levels.