Maturity Classification of “Hupingzao” Jujubes with an Imbalanced Dataset Based on Improved MobileNet V2

Abstract

Fruits with various maturity levels coexist among the harvested jujubes, and have different tastes and uses. Manual grading has a low efficiency and a strong subjectivity. The number of “Hupingzao” jujubes between different maturity levels is unbalanced, which affects the performance of the classifier. To solve the above issue, the class balance loss (CB) was used to improve the MobileNet V2 network, and a transfer learning strategy was used to train the model. The model was optimized based on the selection of an optimizer and learning rate. The model achieved the best classification results using the AdamW optimizer and a learning rate of 0.0001. The application of transfer learning and class balance loss improved the model’s performance. The precision was 96.800~100.000%, the recall was 95.833~100.000%, and the F1 score was 0.963~1.000. To compare the CB-MobileNet V2 performance, the CB-AlexNet, CB-GoogLeNet, CB-ShuffleNet, CB-Inception V3, CB-ResNet 50, and CB-VGG 16 with transfer learning were used to build classification models. Achieving a validation accuracy of 99.058%, and a validation loss value of 0.055, the CB-MobileNet V2 model showed a better overall performance compared with other models. The maturity detection system of “Hupingzao” jujubes was developed to test the model. The testing accuracy of the CB-MobileNet V2 model was 99.294%. The research indicates that the CB-MobileNet V2 model improves the performance of maturity classification, and provides a theoretical basis for intelligent classification of the quality of “Hupingzao” jujubes.

1. Introduction

“Hupingzao” jujubes (Zizyphus jujuba Mill.) can be eaten fresh, made into dried jujubes, or used as a raw material for many functional foods, due to their high nutritional value and medicinal function [1]. The maturity of the fruit is affected by many factors (such as light and flowering time). Fruits of various maturity levels coexist among the harvested “Hupingzao” jujubes. There are differences in the component content of jujubes at different maturity levels, which leads to different tastes and uses for the fruit. In addition, the postharvest respiration intensity and the rate of water loss and weight loss are different when the ripeness period of fresh jujubes is different. The higher the maturity, the higher the respiration rate. The lower the maturity, the faster the water loss [2,3]. The maturity directly affects the shelf life and the storage conditions of fresh jujubes. Therefore, maturity is an important indicator in the grading and sorting of fresh jujubes, and thus has important significance for improving the added value of products and meeting the needs of processing, sales, and storage.

The peel color is an important feature for evaluating the ripeness of fresh jujubes [4,5]. The manual grading of the maturity of jujubes using visual methods is time-consuming and laborious, and is highly intensive with a low efficiency. Computer vision technology intuitively obtains the geometric structure and the appearance characteristics of samples [6,7]. This technology has the advantages of being noncontact, fast, nondestructive, and low-cost, and has been widely used in fruit quality detection and grading. Image processing is a core component of machine vision, and statistical analysis is one of its important theoretical foundations. In the current research, two image processing technologies, mainly based on the traditional machine learning and deep learning techniques, were used.

Image processing technology using the traditional machine learning technique has received much attention in the research on maturity classification for fruits. Based on the extracted color features of apricots, Khojastehnazhand et al. [8] used linear and quadratic discriminant analyses to classify maturity, with accuracies of 0.904 and 0.923, respectively. Wan et al. [9] realized an average accuracy of 99.31% for tomato maturity classification, using the color feature values and the backpropagation neural network. In the classification of the maturity of oil palm fresh fruit bunches, using the principal component analysis method, a data dimensionality reduction based on the color and texture features was performed [10]. The accuracy using an artificial neural network with a backpropagation algorithm was 98.3%. Based on the selected geometric attributes and texture and color features produced by a dimensionality reduction, Azarmdel et al. [11] used artificial neural networks and a support vector machine to classify three maturity stages of mulberry fruits, and the optimal classification accuracy of the prediction set was 99.1%. In the above research, the models established by machine learning achieved good prediction performances. However, in all cases, the required feature information was extracted manually from a small sample set. In the process of feature extraction, the number of features that can be generated is large. If the number of features exceeds a critical value, the performance of the classifier will decrease. For high-dimensional data, a dimensionality reduction process needs to be performed. This method is only suitable for specific objects, has a low efficiency, and a weak repeatability.

Deep learning automatically learns the features from the dataset, and the structure is flexible. In this method, the linear and nonlinear features are mined by effectively integrating the information between channels to adapt to the different learning strategies. The convolutional neural network (CNN) is a deep learning method that includes nonlinear transformation functions. CNN shares the convolution kernels and automatically extracts the features. The built model is transferable using CNN. CNN includes a variety of network structures, such as AlexNet [12], VGG 16 [13], ResNet [14], Inception V3 [15], and GoogLeNet [16]. Gulzar et al. [17] improved VGG 16, which was compared with machine learning methods in the classification of seeds, and the proposed model achieved the best results, with an accuracy of 99.9%. Loddo et al. [18] compared the performance of models using several advanced CNN architectures, the proposed SeedNet model was the best, and the CNN models were all better than traditional machine learning methods. Taner et al. [19] designed a CNN model (Lprttnr1) to classify hazelnut varieties, the accuracy of the proposed model was 98.63%. Hamid et al. [20] used MobileNet V2 to classify 14 different levels of seeds, and the accuracies of the training and test sets were 98% and 95%, respectively. CNN has been widely used in the classification of varieties [21,22,23], defects [24], freshness [25,26], and maturity [27,28] of fruits, and achieved good results. However, the classification of maturity for “Hupingzao” jujubes based on CNN is rarely reported.

Transfer learning can shorten the training time, because it can transfer the relevant knowledge from the learned model to the new task model [29,30]. In the classification of the maturity of fruits, Xiang et al. [31] used a CNN based on transfer learning to classify the maturity of mangoes, and the accuracy was 96.72%. Behera et al. [32] performed the classification of papaya maturity using the traditional machine learning and the CNN of transfer learning, and the classification accuracy of the VGG19 based on transfer learning was 100%. The early typical CNN contains relatively deep layers and a network structure with high complexity, which mainly improve the classification accuracy of the model. Meanwhile, network efficiency has become another consideration. The lightweight network [33,34] is designed based on structural simplification or model compression, has a small volume, and increases speed; for example, ShuffleNet [35] and MobileNet [36].

In the actual images collected, problems with an insufficient data scale and unbalanced datasets are inevitable. When the amount of data in one category is obviously more than that in another category, the category with the large amount of data is given priority to learn, and the category with the small amount cannot be fully learned during the CNN model training process [37]. The imbalance of data between the different categories affected the classification performance of the CNN training model [38,39,40].

Therefore, this study will establish CNN classification models with different network structures based on class-balanced loss (CB) and transfer learning to achieve an efficient and accurate classification of maturity for “Hupingzao” jujubes. The main objectives of this research include: (1) To establish a MobileNet V2 lightweight model with a class-balanced loss (CB-MobileNet V2). (2) To discuss the effect of learning rates and optimizers on the performance of the CB-MobileNet V2 model based on transfer learning. (3) To analyze the impact of CB on the performance of the MobileNet V2 model, and compare the performance of CNN models with different network structures. (4) To develop a detection system for ‘Hupingzao’ jujube maturity, and test the performance of the model to realize a robust and accurate classification.

2. Materials and Methods

2.1. Image Acquisition and Dataset Distributions

According to grades of fresh Chinese jujube fruit (GB/T 22345-2008), the maturity of jujubes is divided into white maturation, crisp maturation, and full maturation due to their different uses. The peel of immature fresh jujube is green. During the white maturation stage, the color of the peel gradually turns from green to white. The chlorophyll content of the exocarp and mesocarp is reduced. The mesocarp is fleshy, and the meat of the fruit is loose. The fruit has less juice, and a low sugar content. It is suitable for processing into candied jujube. Crisp maturation is the period when the fruit turns full red from the stem to the shoulders. The flesh is greenish-white or milky-white, with a high sugar content, rich juice, and a crisp texture. Jujubes in this period are suitable for eating fresh. Combined with market demands, crisp maturation is divided into early-red maturity and half-red maturity. Compared with half-red maturity, early-red maturity jujubes have a smaller colored area, a lower sugar content, and a poor taste. During the full maturation process, the peel is all red, the color is deepened, and the flesh becomes soft. In this study, the samples were collected from an orchard in Taigu, China. Samples of “Hupingzao” jujubes at five maturity levels (immaturity, white maturity, early-red maturity, half-red maturity, and full maturity) are shown in Figure 1.

The images of the samples were collected using an FRD-AL00 camera with a resolution of 2976 × 3968 pixels. The background of the captured image is black, and the light source is natural light. To match the input requirement of the CNN training, the image size was cropped and scaled to 224 × 224 pixels. The collected samples of ‘Hupingzao’ jujubes in the immature, white mature, early-red mature, half-red mature, and full mature periods are 618, 483, 506, 1268, and 1368, respectively. In total, 4243 images of the samples were acquired. The images of each category were randomly divided into a training set, validation set, and test set with a ratio of 7:2:1, respectively. The specific results are shown in

Table 1. Division of datasets.

Maturity	Dataset	Training Set	Valid Set	Prediction Set
Immaturity	618	432	124	62
White maturity	483	338	96	49
Early-red maturity	506	354	101	51
Half-red maturity	1268	888	254	126
Full maturity	1368	957	274	137
Total	4243	2969	849	425

.

2.2. Model Selection

In recent years, CNN has attracted more and more attention, and is widely used in the field of image processing. Due to its good performance and huge economic potential, CNN has been applied in agriculture. CNNs have a variety of popular network structures, such as AlexNet, VGG 16, ResNet, Inception V3, GoogLeNet, ShuffleNet, and MobileNet V2. Convolution operations are the main contribution of computer vision tasks. However, when the network structure becomes deeper and larger, the calculation costs are higher and the running speed is slower, such as in AlexNet, Inception V3, ResNet 50, VGG 16, etc. MobileNet V2 introduced an inverted residual structure and linear bottleneck structure, which reduced the calculation of the convolution. Compared with GoogLeNet and ShuffleNet, MobileNet V2 has been preferred due to its simple architecture and memory-efficient characteristics.

2.3. CB-MobileNet V2

In this study, the normalization method was used to preprocess the data. The CNN contains many parameters, and a large amount of computation is required in the training process. To compensate for these limitations, the transfer learning was introduced. In the transfer learning of this study, the structure of the convolutional layer of the model was left unchanged, a fully connected layer for new tasks was designed to replace the original fully connected layer. A new convolutional network model was formed with the new fully connected layer and the previous convolutional layer. During training, the weights of the feature extraction layers were frozen, and only the weights of the fully connected layers were updated. The transfer learning process used a pretrained deep network as a feature extractor. The pretrained model was successfully trained on the ImageNet dataset with 1000 different classes to complete the classification task, and the trained parameters and weights were obtained. Due to the strong contrast between the images of these datasets and those of “Hupingzao” jujube, the features extracted by the network may be unsuitable for classifying “Hupingzao” jujubes of different maturity. To overcome this shortcoming, fine-tuning the parameters combined with transfer learning was used for training on “Hupingzao” jujube images with different maturity levels. To improve the classification performance of the model on imbalanced datasets, the class-balanced loss was applied to reweight the loss function. The model was tuned on the validation set, and applied to the test set to output classification results. The flowchart of CB-MobileNet V2 is shown in Figure 2.

2.3.1. Class Balance Loss

Data imbalance between different classes is one of the factors that affects the performance of deep learning models. The amount of data is unbalanced among the “Hupingzao” samples with different maturity levels. The performance of the classifier is biased toward the large number of samples, which affects the detection results of the deep learning model. Therefore, the loss function was weighted and improved based on the class balance loss in this study, and the loss was rebalanced based on the number of valid samples of each class to improve the detection accuracy. Class-balanced loss (CB) [41] used a weighting factor to solve the training problem of imbalanced data, which can be expressed as:

$$\mathrm{CB}(P,y)=\frac{1}{E_{n_y}}L(P,y)=\frac{1-\beta }{1-{\beta }^{n_y}}L(P,y)$$

(1)

where n_y represents the number of samples in the true class y, En is the valid number of samples, P is the probability of model’s predicted class, P = [p₁, p₂,..., p_C]^T, C is the total number of classes and p_i ∈ [0,1]. The loss function is defined as L(P, y). The hyperparameter β is used to adjust the class balance term and β ∈ [0,1].

Based on the focal loss (FL) [42], a class-balanced focal loss (CB_focal) is formed by adding a class-balanced term. FL and CB_focal are represented as follows:

$$\mathrm{FL}(\mathrm{z},y)=-{{\displaystyle \sum _{i=1}^C(1-p_i^t)}}^{\gamma }\log (p_i^t)$$

(2)

$${\mathrm{CB}}_{\mathrm{focal}}(z,y)=-\frac{1-\beta }{1-{\beta }^{n_y}}{{\displaystyle \sum _{i=1}^C(1-p_i^t)}}^{\gamma }\log (p_i^t)$$

(3)

where γ is the focusing parameter, z is the prediction output of the model, p_i^t = sigmoid (z_i^t) = 1/(1 + exp(−z_i^t)). If i = y, then z_i^t = z_i. Otherwise, z_i^t = −z_i.

2.3.2. MobileNet V2

MobileNet is a lightweight neural network, the core is a depthwise separable convolution block. The block of depthwise separable convolution is composed of a 3 × 3 depthwise separable convolution and a 1 × 1 pointwise convolution. MobileNet V2 [43] contains a total of 28 layers, and a residual connection is added to a specific block. In order to prevent the loss of information in the nonlinear layer, a linear bottleneck layer is introduced. The separable convolution is applied to the residual structure to form an inverted residual block. The data are first dimensionally increased, and then dimensionally reduced. The memory footprint required for training is reduced.

2.4. Experimental Environment Settings and Model Evaluation Indicators

In this study, Python 3.7.0, Pytorch 1.7, CUDA, and CUDNN libraries were used for training. In the training process, the operating system was configured as Windows 10, and a GPU of NVIDIA RTX 2060 was used to speed up model training. CNN models were carried out using an Anaconda3 software. Accuracy, precision, recall, and F1 score were used as evaluation indicators for model performance. The specific calculation method is as follows:

$$\mathrm{Accuracy}=\frac{TP+TN}{TP+TN+FN+FP}$$

(4)

$$\mathrm{Precision}=\frac{TP}{TP+FP}$$

(5)

$$\mathrm{Recall}=\frac{TP}{TP+FN}$$

(6)

$$\mathrm{F}1\mathrm{Score}=2\times \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}$$

(7)

where TP and FN are the number of positive samples predicted as positive and negative classes, respectively, and FP and TN are the number of negative samples predicted as negative and positive classes, respectively.

3. Results and Discussion

3.1. Influence of Learning Rate and Optimizer on Models

Learning rate (LR) is an important hyperparameter of CNN, and is related to the convergence (such as the local minimum, or convergence speed) of objective function. The optimizer calculates and updates the parameter values of model training and output, so that the network model continuously approaches the optimal value. The optimizer is used to minimize (or maximize) the loss function and improve accuracy. In this study, the CB-MobileNet V2 network combined with transfer learning was used to build the classification models for the maturity of “Hupingzao” jujubes. The Adam, AdamW, ASGD, and SGD optimizers were used for model training, respectively. The batch size was 16, and the number of epochs was 100. The dropout function was used to reduce overfitting, and was set to 0.5. The learning rates were set as 0.01, 0.001, 0.0001, and 0.00001 for model training, respectively. The validation accuracy and loss values of the four optimizers are shown in Figure 3, Figure 4, Figure 5 and Figure 6, respectively.

In Figure 3 and Figure 4, the validation accuracy and loss curves with LRs of 0.0001 and 0.00001 were close, respectively. The convergence of the curves were stable. With an LR of 0.001, it was seen that the validation accuracy and loss curves oscillated greatly. In Figure 5 and Figure 6, the convergence of the validation accuracy and loss curve with the three LRs were all stable. The loss curves for LRs of 0.001 and 0.0001 were similar, and lower than the LR of 0.00001. For the validation accuracy curve in Figure 5, the LR of 0.0001 was slightly higher than LRs of 0.001 and 0.00001. In Figure 6, the validation accuracy curves of the three LRs were similar, but the LR of 0.001 had slightly larger fluctuations at individual epochs. The four optimizers showed a good fitting ability. On the whole, when a larger LR was adopted, the model was easily able to generate larger shocks. The LR became smaller, the convergence speed of models slowed down, and the time to find the optimal value increased. When Adam, AdamW, ASGD, and SGD optimizers used learning rates of 0.01, 0.001, 0.0001, and 0.00001, the optimal validation results are shown in

Table 2. Optimal results using different learning rates and optimizers.

Optimizer	Learning Rate	Loss Value of Train	Accuracy of Train	Loss Value of Validation	Accuracy of Validation
Adam	0.001	0.016	99.630%	0.055	98.704%
	0.0001	0.005	99.966%	0.018	98.822%
	0.00001	0.004	99.966%	0.015	98.587%
AdamW	0.001	0.037	99.394%	0.131	98.469%
	0.0001	0.016	99.630%	0.055	99.058%
	0.00001	0.062	99.293%	0.217	98.587%
ASGD	0.001	0.127	98.585%	0.444	98.351%
	0.0001	0.079	99.394%	0.276	98.704%
	0.00001	0.486	95.386%	1.704	97.880%
SGD	0.001	0.123	98.080%	0.432	98.822%
	0.0001	0.023	99.798%	0.082	98.940%
	0.00001	0.123	99.023%	0.431	98.469%

.

Combining Figure 3, Figure 4, Figure 5 and Figure 6 and Table 2, it was seen that an LR of 0.0001 performed better than 0.001 and 0.00001 LRs for validation results under the same training conditions. According to the validation results, the AdamW was better than Adam, ASGD, and SGD at an LR of 0.0001. The convergence speed of the model and the accuracy of detection were affected by different optimizers and learning rates. To analyze the classification for each maturity, the precision, recall, and F1 scores based on models using different optimizers and learning rates were calculated. The classification results of the 12 combined models for each maturity are shown in Figure 7. Each optimizer achieved the best classification performance at an LR of 0.0001. The precision, recall, and F1 scores were above 93.939%, 94.792%, and 0.948, respectively. At an LR of 0.0001, AdamW had the highest precision, recall, and F1 score for each maturity among the four optimizers. Among the 12 combinations, the AdamW optimizer with an LR of 0.0001 achieved the best validation result. The validation loss and accuracy were 0.055 and 99.058%, respectively. Therefore, the AdamW optimizer and a learning rate of 0.0001 were selected for training the classification model of “Hupingzao” jujube maturity.

3.2. Comparison of Model Performance

To evaluate the performance of CB-MobileNet V2 in the maturity classification of “Hupingzao” jujubes, ablation experiments were performed and different network models were compared in this study. The batch size, number of epochs, and dropout were 16, 100, and 0.5, respectively. AdamW was used as an optimizer, and the learning rate was 0.0001.

3.2.1. Ablation Study

To realize the lightweight of the model, the MobileNet V2 was used as the base model, four ablation schemes were used to perform model training. The protocol and results of the ablation study are shown in

Table 3. Results of ablation experiment.

Class Balance	Transfer Learning	Loss Value of Train	Accuracy of Train	Loss Value of Validation	Accuracy of Validation
		1.202	55.040%	1.152	57.597%
	With	0.107	96.362%	0.064	98.233%
With		0.025	99.630%	0.0873	97.880%
With	With	0.016	99.630%	0.055	99.058%

. Compared with the base model of MobileNet V2, the validation accuracy was increased by 40.636% using only the transfer learning strategy. Adding only the class balance increased the validation accuracy of the model by 40.283%. The addition of both class balance and transfer learning improved the validation accuracy by 41.461%. Transfer learning and class balance loss both improved model accuracy. The highest validation accuracy was obtained using both class balance and transfer learning, with a 0.825% improvement in validation accuracy compared to using only transfer learning.

It was not comprehensive enough to use accuracy alone to evaluate model performance, because the dataset was unbalanced. It was possible that the model had a high accuracy, while a class with a small amount of data had a high classification error rate. To visually evaluate the performance of models, the confusion matrices using four protocols are shown in Figure 8.

In Figure 8, each row represents an actual class, and each column represents a predicted class. The diagonal area indicates that the actual and predicted classes are consistent. When the model was trained using the strategy without transfer learning, the number of the samples that were correctly predicted increased significantly after the addition of class balance. The ability of classification for immature and white maturity samples was significantly improved. When the model was trained using the strategy of transfer learning, it was easy to confuse the immaturity samples and the white maturity samples before adding the class balance terms. After adding the class balance term, the number of white maturity samples that were wrongly predicted as immaturity samples was reduced by six samples. Additionally, the number of half-red maturity samples that were wrongly predicted as full maturity samples was reduced by two samples. The precision, recall, and F1 score of each class were calculated according to the confusion matrix using transfer learning. The results are shown in

Table 4. Effect of class balance loss on validation results.

Category	Class Balance	Precision	Recall	F1 Score
Immaturity	Without	92.366%	97.581%	0.949
	With	96.800%	97.581%	0.972
White maturity	Without	96.629%	89.583%	0.930
	With	96.842%	95.833%	0.963
Early-red maturity	Without	100.000%	100.000%	1.000
	With	100.000%	100.000%	1.000
Half-red maturity	Without	100.000%	99.213%	0.996
	With	99.608%	100.000%	0.998
Full maturity	Without	99.275%	100.000%	0.996
	With	100.000%	99.635%	0.998

.

Among the validation results of the model without the class balance item in Table 4, the white maturity sample was the worst. The recall and F1 score were 89.583% and 0.929, respectively. After adding the class balance term, the established model achieved good validation results for the samples of the five maturity levels. The precision was 96.800~100.000%, the recall was 95.833~100.000%, and the F1 score was 0.963~1.000. In the validation results for the five categories, white maturity samples showed the most improvement. The recall was improved by 6.25%, and the F1 score was improved by 0.033. This shows that the CB exhibited good performance for the class with a small amount of data. The precision and F1 score of the immaturity sample were improved by 4.434% and 0.023, respectively. The recall and F1 score of the half-red maturity sample were improved by 0.787% and 0.002, respectively. The precision and F1 score of the full maturity sample were improved by 0.725% and 0.002, respectively. For the early-red maturity sample, the precision, recall, and F1 score were the same before and after adding the class balance term. For the remaining four maturity samples, the F1 scores were all improved after adding the class balance item. This indicated that the application of CB improved the performance of the “Hupingzao” jujube maturity detection model.

In order to visually demonstrate the effectiveness of the CB-MobileNet V2 model combined with transfer learning, Grad-CAM++ technology was used to build the class activation map of the samples in this study. Based on the model before and after adding the class balance term, the class activation maps of the five maturity “Hupingzao” jujubes are shown in Figure 9.

Before adding the class balance term in Figure 9, the sample localization was not sufficiently accurate. For immaturity, white maturity, early-red maturity, and half-red maturity samples in feature extraction, only part of the region was focused. The features extracted from samples of the five categories contained a lot of background interference information. After adding the class balance item, the subject position of the sample could be accurately located, and the background interference information was significantly reduced. This shows that the model, after adding the class balance term, could effectively extract the key information of the samples, obviously improve the interference of the background area, and improve the classification performance.

3.2.2. Model Comparison

To compare the performance of the CB-MobileNet V2 network in classifying the maturity of “Hupingzao” jujube, AlexNet, Inception V3, ResNet 50, VGG 16, GoogLeNet, and ShuffleNet networks were selected to compare the performance of the models. For fair comparison, the basic architecture of these deep learning models was left unchanged. The above network was also improved using the class-balanced loss, and all models were trained using a transfer learning strategy. The optimal validation results of the seven models are shown in

Table 5. Optimal results using different networks.

Model	Loss Value of Train	Accuracy of Train	Loss Value of Validation	Accuracy of Validation
CB-AlexNet	0.047	98.787%	0.165	98.587%
CB-MobileNet V2	0.016	99.630%	0.055	99.058%
CB-GoogLeNet	0.001	100.000%	0.004	98.940%
CB-VGG 16	0.001	100.000%	0.001	98.704%
CB-Inception V3	0.001	100.000%	0.067	98.704%
CB-Resnet 50	0.002	100.000%	0.015	98.940%
CB-ShuffleNet	0.010	99.798%	0.035	98.587%

.

For the seven models, the validation loss was 0.001~0.165, the validation accuracy was 98.587~99.058%, and the difference in accuracy between the training set and the validation set was 0.200~1.296%. The seven models all exhibited high accuracy. The CB-MobileNet V2 achieved the highest validation accuracy. The precision, recall, and F1 scores of models for each class were calculated. The results are shown in Figure 10.

The precision, recall, and F1 scores were above 93.069%, 90.625%, and 0.941, respectively. Figure 10 shows that CB-MobileNet V2 was, overall, higher than the other models for precision, recall, and F1 score curves. The CB-MobileNet V2 model achieved good recall and precision for each maturity category. However, there was either low recall or precision in some of the categories when other models were used. For the recall of the white maturity samples, the CB-AlexNet (90.625%), CB-VGG 16 (91.667%), CB-ResNet 50 (91.667%), CB-ShuffleNet (92.708%), and CB-GoogLeNet (94.791%) models were worse than the CB-MobileNet V2 and CB-Inception V3 models. The CB-Inception V3 model had the lowest recall of immature samples compared to the other models. The CB-AlexNet and CB-Inception V3 models had the lowest precision for immature and white mature samples, respectively. In addition, the CB-MobileNet V2 achieved the highest F1 score for each maturity. Therefore, CB-MobileNet V2 showed the best classification performance. The validation loss and accuracy were 0.055 and 99.058%, respectively.

3.3. Test of Model Performance

To test the detection effect and practicability of the CB-MobileNet V2 model, a “Hupingzao” jujube maturity detection system was developed. The development environment was PyQt5 and Python 3.7.0 (Python Software Foundation, Delaware, USA). The front end of the system is shown in Figure 11. The front–end interface included data collection, run, and stop buttons, and displayed the classification result and discrimination accuracy of the corresponding sample. The background of this system could call the OpenCV camera to obtain images using the written Python language, and the trained model was called to classify the maturity of jujubes. For the example shown in Figure 11, the sample was identified as half-red maturity with an accuracy of 0.9994. The collected images were tested based on this system. The results are shown in

Table 6. Test results for samples of different maturity levels.

Category	Precision	Recall	F1 Score
Immaturity	98.387%	98.387%	0.984
White maturity	97.917%	95.918%	0.969
Early-red maturity	98.077%	100.000%	0.990
Half-red maturity	100.000%	100.000%	1.000
Full maturity	100.000%	100.000%	1.000

.

In Table 6, the precision of immaturity, white maturity, early-red maturity, half-red maturity, and full maturity samples were 97.917~100.000%. The recall was 95.918~100.000%, and the F1 score was 0.969~1.000. Based on this system, the testing accuracy of the CB-MobileNet V2 model was 99.294%. This shows that the model achieved good classification performance for samples of all five maturity levels.

In the maturity detection of jujubes based on traditional image processing and machine learning methods, classification models were established by extracting features (such as color) [44,45,46]. The classification ability relied on the handmade features and the learned classifiers. This method is mainly used to solve the learning problem for limited samples, and has a weak generalization ability and repeatability. CNN reduces the complexity of artificially extracting features because it automatically and hierarchically performs feature learning from the dataset.

At present, the maturity classification of “Hupingzao” jujubes using CNN has not been reported. In the maturity classification of other varieties of date fruits, Nasiri et al. [47] carried out the four maturity classifications for Shahani dates using a CNN-created model based on VGG 16, and the overall classification accuracy was 96.98%. Pérez et al. [48] performed two maturity classifications on Medjool dates, and a VGG 19 based on transfer learning obtained an optimal accuracy rate of 99.32%. Faisal et al. [49] used ResNet to detect five maturity levels of date fruits with an accuracy of 99.05%. Because of the different characteristics and uses of jujubes, the classification model built with other varieties of jujube was difficult to apply to the classification of “Hupingzao” jujubes.

CNN requires images of large orders of magnitude, which requires high cost and effort. In this study, the validation accuracy increased by 40.636% using the transfer learning strategy. Transfer learning can alleviate the dependence of deep learning on data to a certain extent, and improve training efficiency. Data imbalance between different classes is a limiting factor affecting the performance of the classification model. Adding class balance increased the validation accuracy of the model by 40.283%. Adding both class balance and transfer learning, the validation accuracy improved by 41.461%. The immature and white mature samples were also easy to miscalculate when only the transfer learning strategy was used. After adding the class balance loss to the model with transfer learning, the recall increased from 89.583 to 95.833%, and the F1 score increased from 0.930 to 0.963, for white maturity. More accurate positioning in the extraction of sample features and less background interference information for the five maturity samples was realized. Through the application of class balance loss and transfer learning, the problem of insufficient learning from a class with a small sample size in the dataset improved. Using 12 different combinations, the AdamW optimizer with a learning rate of 0.0001 achieved the best validation result; the validation accuracy was 99.058%. In comparison with other models, CB-MobileNet V2 with transfer learning achieved the best performance. The CB-MobileNet V2 model achieved an accuracy of 99.294% in a test based on the developed detection system. Therefore, the CB-MobileNet V2 network with transfer learning demonstrated good comprehensive performance. This research has realized the classification of maturity for “Hupingzao” jujubes, and has certain practical significance and application value for solving the quality classification of jujubes.

4. Conclusions

In this study, class balance loss was used to improve the MobileNet V2 network, and a CB-MobileNet V2 model with transfer learning was used to classify the maturity of “Hupingzao” jujubes. The optimizer and learning rate affected the performance of the model. Under the same training conditions, the model with a learning rate of 0.0001 achieved the best classification result. At a learning rate of 0.0001, the AdamW optimizer was better than Adam, ASGD, and SGD optimizers for classification results. Data imbalance among classes in datasets affected the performance of models. The application of class balance loss and transfer learning improved the performance of the MobileNet V2 model, and classification accuracy and F1 score were increased. Compared to MobileNet V2 based on transfer learning, the white maturity sample had the highest improvement in the validation results among samples of the five maturity levels using CB-MobileNet V2. The recall and F1 score were increased by 6.25% and 0.033, respectively. Class balance loss performed well in discriminating classes with a small sample size. Compared with CB-AlexNet, CB-GoogLeNet, CB-ShuffleNet, CB-Inception V3, CB-ResNet 50, and CB-VGG 16 network models with transfer learning, the CB-MobileNet V2 model achieved better classification results. The validation loss and accuracy were 0.055 and 99.058%, respectively. The precision was 96.800~100.000%, the recall was 95.833~100.000%, and the F1 score was 0.963~1.000. A maturity detection system of “Hupingzao” jujubes was developed, and the testing accuracy of the CB-MobileNet V2 model was 99.294%. The testing precision, recall, and F1 score were 97.917~100.000%, 95.918~100.000%, and 0.969~1.000, respectively. The CB-MobileNet V2 model exhibited a good overall performance. Therefore, this study achieved the maturity classification of “Hupingzao” jujubes. In the future, we plan to develop a multitask classification model for the maturity and defects of jujubes at the same time, and to develop software and hardware for automatic grading equipment for their comprehensive classification.