Next Article in Journal
Impact of Circular Brewer’s Spent Grain Flour after In Vitro Gastrointestinal Digestion on Human Gut Microbiota
Next Article in Special Issue
Effects of Variations in the Chemical Composition of Individual Rice Grains on the Eating Quality of Hybrid Indica Rice Based on Near-Infrared Spectroscopy
Previous Article in Journal
Ham Tourism in Andalusia: An Untapped Opportunity in the Rural Environment
Previous Article in Special Issue
Miniaturized NIR Spectroscopy in Food Analysis and Quality Control: Promises, Challenges, and Perspectives
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Foods
Volume 11
Issue 15
10.3390/foods11152278
foods-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Evaluation of Mutton Adulteration under the Effect of Mutton Flavour Essence Using Hyperspectral Imaging Combined with Machine Learning and Sparrow Search Algorithm

by
Binbin Fan
1,
Rongguang Zhu
1,2,*,
Dongyu He
1,
Shichang Wang
1,
Xiaomin Cui
1 and
Xuedong Yao
1
1
College of Mechanical and Electrical Engineering, Shihezi University, Shihezi 832003, China
2
Key Laboratory of the Ministry of Agriculture and Rural Affairs, Shihezi University, Shihezi 832003, China
*
Author to whom correspondence should be addressed.
Foods 2022, 11(15), 2278; https://doi.org/10.3390/foods11152278
Submission received: 12 July 2022 / Revised: 27 July 2022 / Accepted: 27 July 2022 / Published: 30 July 2022
(This article belongs to the Special Issue Vibrational Spectroscopy, Chemometrics and Molecular Profiles: Applications to the Quality Assessment of Foodstuffs)
Browse Figures
Versions Notes

Abstract

:
The evaluation of mutton adulteration faces new challenges because of mutton flavour essence, which achieves a similar flavour between the adulterant and mutton. Hence, methods for classifying and quantifying the adulterated mutton under the effect of mutton flavour essence, based on near-infrared hyperspectral imaging (NIR-HSI, 1000–2500 nm) combined with machine learning (ML) and sparrow search algorithm (SSA), were proposed in this study. After spectral preprocessing via first derivative combined with multiple scattering correction (1D + MSC), classification and quantification models were established using back propagation neural network (BP), extreme learning machine (ELM) and support vector machine/regression (SVM/SVR). SSA was further used to explore the global optimal parameters of these models. Results showed that the performance of models improves after optimisation via the SSA. SSA-SVM achieved the optimal discrimination result, with an accuracy of 99.79% in the prediction set; SSA-SVR achieved the optimal prediction result, with an RP2 of 0.9304 and an RMSEP of 0.0458 g·g−1. Hence, NIR-HSI combined with ML and SSA is feasible for classification and quantification of mutton adulteration under the effect of mutton flavour essence. This study can provide a theoretical and practical reference for the evaluation and supervision of food quality under complex conditions.

1. Introduction

Mutton is a widely consumed meat around the world because of its high content of vitamins, proteins and trace elements, such as calcium, phosphorus and iron, which are needed by the human body [1,2,3]. Some unscrupulous merchants adulterate mutton with pork and additives to increase their profits because of the high price of mutton. Mutton flavour essence is a representative additive in the market that provides pork with a strong mutton flavour. This essence will release benzene, phenol and other toxic substances in the heating process that can cause serious harm to the health of consumers [4,5]. At the same time, the mutton flavour essence used for adulteration will seriously interfere with common discriminant results. At present, studies on the detection of mutton adulteration with pork under the effect of mutton flavour essence are lacking. This phenomenon harms both the interests and physical health of consumers [6]. Therefore, the evaluation of mutton adulteration with pork under the effect of mutton flavour essence is increasingly important.
Existing detection methods of meat adulteration mainly include sensory test, chromatographic analysis [7,8], immune analysis [9,10], DNA analysis [11,12] and electronic nose/taste [13,14]. Sensory tests have been unable to meet the need whilst other detection methods present high technical requirements and complex operation with the improvement in adulteration methods. In recent years, spectral technology has been widely used in adulteration detection of meat and meat products due to its simple and fast operation and non-destructive characteristics [15,16]. Near-infrared hyperspectral imaging (NIR-HSI) is a three-dimensional information acquisition technology combining spectral and image technology [17]. NIR-HSI has been applied to detect food adulteration, in particular, for meat products [18,19], such as pork lung in pork [20], jowl meat in pork [21] and ternary adulteration [22]. These reports mainly focus on pure meat adulteration. A variety of food additives are also mixed into meat to interfere with the detection because of the diversification of methods for meat adulteration. Note that meat detection under the effect of food additives has been reported. Nunes et al. [23] detected adulterated beef with multiple water-retaining agents based on mid-infrared spectroscopy and successfully distinguished adulterated beef. However, the detection of mutton adulteration under the effect of mutton flavour essence by hyperspectral means has not been reported. Hence, NIR-HSI was used to detect the adulteration of pork under the effect of mutton flavour essence in this paper.
The data analysis and modelling mainly depend on machine learning (ML) methods. Support vector machine (SVM), back propagation neural network (BP) and extreme learning machine (ELM) have been widely used in the detection of meat and adulteration [16,24,25,26,27]. The uses of new algorithms to optimise parameters and their combination with practical problems have become an important research direction for machine learning in recent years [28,29,30]. Quantum particle swarm optimisation (QPSO) was used to improve SVM in evaluating meat freshness [31]. An improved AdaBoost-BP model was applied to the rapid detection of mutton freshness [32]. Genetic algorithm (GA) was utilised to improve ELM for the identification of mutton parts [33]. Although these algorithms improve the performance of the model, they still present limitations, such as falling into the local optimal solution.
A swarm intelligence optimisation algorithm, named sparrow search algorithm (SSA), was proposed in 2020 to overcome the problem of traditional algorithms falling into the local optimum [34]. Compared with other common algorithms, SSA demonstrates the advantages of high precision, fast convergence, enhanced stability and robustness [34]. SSA was used for the optimisation of the model for detection [35,36,37]. SSA was used to optimise BP combined with hyperspectral means to detect protein content in milk rapidly; the results showed that its performance was superior to that of other methods [38]. However, the application of the SSA improved the model in meat adulteration, though detection remains unverified. Therefore, ML models are optimised by SSA for classification and quantification of adulterated mutton under the effect of mutton flavour essence.
In this study, the feasibility of near-infrared hyperspectral imaging combined with machine learning and sparrow search algorithm for evaluating adulterated mutton with pork under the effect of mutton flavour essence was explored. This study aims to: (1) acquire the spectral information of mutton, pork and adulterated mixture under the effect of mutton flavour essence using NIR-HSI; (2) establish classification models based on BP, ELM and SVM; (3) establish quantification models based on BP, ELM and SVR; (4) use SSA to optimise parameters in the classification and quantification models and select the optimum models by comparing the classification and quantification results.

2. Materials and Methods

2.1. Sample Preparation

Fresh mutton from the hind leg and fresh pork from three parts (front leg, hind leg and back) were used as meat sources in this study. The addition of different parts of pork can expand the scope of application and improve the generalization of the model. All meat was purchased from a local supermarket in Shihezi City, Xinjiang, and met national quarantine standards. The experimental meat was transported to the laboratory in an incubator at 4 °C. The processed meat was minced and mixed evenly with a meat-mincing machine (JYS-A900, Joyoung Co., Ltd., Zhejiang, China) for 30 s after the removal of evident fat and skin. The mutton flavour essence (Xianghaisheng Co., Ltd., Shandong, China) was weighed at a dosage of 3 g per kilogram of pork according to food safety regulations and then added into distilled water to prepare the solvent with a concentration of 0.05 g/mL. After that, the minced pork was added into the solvent and soaked for 20 min to mix fully. Then, the minced pork with mutton flavour essence was obtained after removing residue solvents on the surface. The minced pork from three parts with mutton flavour essence was mixed into the minced mutton in different proportions (5, 10, 15, 20, 30 and 40%) by weight to prepare the adulterated mutton samples. Fully mixed samples were placed in a Petri dish with a diameter of 6 cm for compaction and samples with smooth surface were obtained. A total of 144 adulterated samples were prepared for quantification, with 8 samples of each proportion in each part (8 × 6 × 3 = 144). Meanwhile, 24 pork samples from three parts (8 samples in each part) and 24 mutton samples were prepared using the same method. In total, 192 samples (24 + 24 + 144 = 192) in three categories were prepared for qualification. The prepared samples were stored in a refrigerator at −5 °C until detection. A diagram of sample preparation process is shown in Figure 1a.

2.2. Acquisition and Calibration of NIR-HSI Images

A liner scanning near-infrared hyperspectral imaging system was used in this study. It can produce 288 wavelengths in a spectral range of 1000–2500 nm. Figure 1b shows that the system consists of a spectrograph (ImSpector N25E 2/3, Spectral Imaging Ltd. Oulu, Finland), a CCD camera (Zephir-2.5-320, Photon Ltd., Montreal, Canada), a moving stage driven by a stepper motor, lighting sources (150 W), a computer and a monitor. Light intensity, exposure time, speed of the moving stage, illuminating angle of the light source and the height of lens were set to 100, 4 s, 62.77 mm/s, 60° and 170 mm, respectively, to obtain the real proportion of image size and high spatial resolution. Samples were placed on black cardboard as the background during the acquisition process. The experiment was carried out at 26 ± 1 °C and 30 ± 5% relative humidity.
The system was preheated for 30 min before image acquisition. Samples were then taken out of the refrigerator and left to warm to room temperature for 5 min. Sample images needed to be corrected with the standard black (with lens cover) and white reference (PTFE standard whiteboard) to reduce noise and external interference factors according to Equation (1):
I = I R − I B I W − I B ,
where I is the calibrated image data, IR is the original hyperspectral image data, IW is the full-white calibration image data and IB is the full-black calibration image data acquired after closing the camera with the lens cover.

2.3. Spectral Acquisition

The meat was separated from the background of the near-infrared hyperspectral image of the sample and the region of interest (ROI) was determined via image segmentation method [39]. ENVI 4.7 (Exelis Visual Information Solutions Ltd., Boulder, CO, USA) was used to process the original hyperspectral image and eliminate the interference. Firstly, backgrounds and shadows of samples were transformed to black using band subtraction and mask method. Fats and bright spots in samples were then highlighted through band addition and removed via mask method and binary processing. The pure muscle part of the samples was obtained after processing. Sample data were expanded to avoid under-fitting or over-fitting caused by insufficient data. Spectral data for each pixel were extracted according to the vertical direction of the ROI and randomly combined and divided into 10 blocks in equal proportion. The average spectrum of each block was extracted as the representative data of the sample. Lastly, 10 spectra were obtained for each sample and 1920 spectra were obtained for 192 samples. A wavelength of 1000–2445 nm was selected for analysis because the spectral noise after 2445 nm was high. The spectral acquisition is shown in Figure 1c.

2.4. Spectral Data Preprocessing

A certain amount of noise and interference information was generated due to external factors during the data acquisition [40]. Hence, preprocessing the original spectra is necessary. Spectral preprocessing can be divided into baseline correction, scattering correction, smoothing and scaling [41]. Baseline correction can eliminate the influence of instrument background or drift on signal. Scattering correction is used to remove the scattering effect caused by uneven distribution of particles. Smoothing can eliminate random noise and scaling can weaken data-scale difference. According to the characteristics of background interference and uneven distribution of sample particles in this study, the method of first derivative (1D) and multiple scattering correction (MSC) was selected as the spectral preprocessing method.

2.5. Model Establishment and Evaluation

2.5.1. Modelling Methods

Back propagation neural network (BP) is a multi-layer feed-forward network algorithm based on error back propagation that presents high self-learning and adaptive ability [42]. It mainly realizes nonlinear mapping between input and output through learning. The number of hidden layers and the number of neurons in each layer can be set according to different situations and the performance varies with the structure.
Extreme learning machine (ELM) is a kind of machine learning method based on feed-forward neural network (FNN) and usually applied to supervised and unsupervised learning [43]. The weight of hidden layer nodes is selected randomly or artificially and does not need to be updated. The learning process presents advantages of strong generalization ability and fast calculation speed because it only calculates the output weight.
Support vector machine (SVM) is a supervised machine learning algorithm that determines the optimal separation hyperplane in the feature space to maximise the sample interval in the calibration set to avoid dimension disaster to some extent [44]. SVM can solve not only classification problems, but also regression problems through its important branch called support vector regression (SVR).

2.5.2. Sparrow Search Algorithm Optimisation

Sparrow search algorithm (SSA) is a swarm intelligence optimisation algorithm for simulating the behaviour of sparrow foraging and evading predators [34]. SSA can effectively avoid falling into the local optimal solution because of its advantages of minimal iterations and high accuracy in the prediction model [45]. SSA was used in this study to optimise parameters of BP, ELM and SVM/SVR models. The process of constructing the optimisation model based on SSA is presented as follows.
Step 1: Initialise population, iterations, proportion of discoverer and participant, calculate and sort the fitness of all sparrows.
Step 2: Update the position of discoverers and participants. Discoverers with satisfactory fitness values can preferentially obtain food in the searching process and provide foraging direction for all participants. The participants will monitor the discoverers and immediately compete with discoverers when the discoverers search for better food in the foraging process. The position of the discoverers in each iteration is updated, as in Equations (2) and (3).
X i , j t + 1 = { X i , j · exp ( − i α · i t e r max ) , i f   R 2 < S T X i , j + Q · L , i f   R 2 ≥ S T
X i , j t + 1 = { Q · exp ( − X w o r s t − X i , j t i 2 ) , i f   i > n / 2 X P t + 1 + | X i , j − X P t + 1 | · A + · L , o t h e r w i s e
where t represents the current iteration number, itermax is the maximum iteration number and j is the constant of [1, d]. Xi,j is the position of the ith sparrow in the j dimension, XP is the best position for discoverers and Xworst is the worst position. The discoverers can search in a safe foraging environment when R2 < ST. The predators appear and all sparrows move to forage rapidly when R2 ≥ ST. The ith participant with low-fitness value travels to other places for food when i > n/2.
Step 3: Update the position of watchmen. Watchmen are randomly generated in the population and generally account for 10% to 20% of the total. The position of the watchmen in each iteration is updated as in Equation (4).
X i , j t + 1 = { X b e s t t + β · | X i , j t − X b e s t t | , i f   f i > f g X i , j t + K · ( | X i , j t − X w o r s t t | ( f i − f w ) + ε ) , i f   f i = f g
where Xbest is the global optimal position; β is the step-size control parameter; fi is the current individual fitness value; and fg and fw are the global best and global worst fitness values, respectively.
In this study, the initial population size was 20; the maximum number of iterations was 100; the proportion of discoverers and participants was 7:3; the proportion of watchmen was set to be 20%. These parameters were determined according to the commonly used parameters and the actual situation in this study [46]. SSA was applied to optimise the model parameters and obtain the optimal solution from the global optimal position. The optimal solution for BP and ELM was obtained through iteration and assigned to weights and thresholds to complete the parameter optimisation of the SSA-BP and SSA-ELM models. The range of weights and thresholds is (−5, 5). The optimal solution for SVM and SVR was obtained through iteration and assigned to the penalty coefficient c and RBF kernel function radius g to complete the parameter optimisation of SSA-SVM and SSA-SVR models, respectively. The range of c and g is (1 × 10−5, 100).

2.5.3. Establishment and Evaluation of Classification Models

In the establishment of classification models, 24 mutton, 24 pork and 48 adulterated mixture samples in each part (24 + 24 + 48 × 3 = 192 samples) were randomly divided into calibration set (144 samples) and prediction set (48 samples) at a ratio of 3:1. As such, 10 average spectra were extracted from each sample region. Hence, calibration and prediction set included 1440 and 450 spectra, respectively. Three classes were set: mutton (CLASS M), pork (CLASS P) and adulterated mixture (CLASS MP).
Firstly, ELM, BP and SVM classification models were established for different classes of samples. Secondly, SSA was used to find the global optimal position as the potential optimal solution of the models. Lastly, the SSA-ELM, SSA-BP and SSA-SVM models were constructed using optimal parameters. The 10-fold cross-validation divided the data set into 10 parts, of which 9 parts were used as training data and 1 part as test data in turn. It is a commonly used method to improve the generalization ability of the model. The 10-fold cross-validation result is taken as the calibration set result.
The performance of the models was evaluated using the discriminant results of each set after establishing the classification models. The discriminant property in the classification model was evaluated according to accuracy and accuracy confusion matrix. Accuracy is the ratio of the correct classification samples to the total samples. High model accuracy and few error classes in the confusion matrix correspond to enhanced model performance.

2.5.4. Establishment and Evaluation of Quantification Models

In the establishment of quantification models, 8 adulterated mixture samples in each proportion of each part (8 × 6 × 3, a total of 144 samples) were randomly divided into calibration set (108 samples) and prediction set (36 samples) at a ratio of 3:1. As such, 10 average spectra were extracted from each sample region. Hence, calibration and prediction set included 1050 and 360 spectra, respectively.
Firstly, ELM, BP and SVM classification models were established for different proportions of samples. Secondly, SSA was used to find the global optimal position as the potential optimal solution of the models. Lastly, the SSA-ELM, SSA-BP and SSA-SVR models were constructed using optimal parameters. The 10-fold cross-validation result is the calibration set result.
The performance of models was evaluated by the regression results of each set. The regression property of the quantification models was evaluated with coefficient of determination (R2) and root mean square error (RMSE). High-R2 and low-RMSE values represent enhanced performance. The software used in this study was MATLAB 2019b (Math Works, Natick, MA, USA).

3. Results and Discussion

3.1. Analysis of Spectral Characteristics

The reflectance spectral curves of all samples are shown in Figure 2a. The spectral trends were similar and spectral absorption peaks mainly appeared at 1020, 1250, 1520 and 2150 nm. Amongst them, the spectral absorption bands at 1020 and 1520 nm were closely related to the stretching vibration of the first-order and second-order frequency doubling characteristics of the N-H bond, located at the functional groups of the protein molecular structure, respectively. The absorption peak at 1250 nm was related to the second-order frequency doubling characteristic in the C-H bond in the molecular structure of meat organic components. The absorption band at 2150 nm was related to the combination frequency characteristic in the O-H bond of molecular structure functional groups in water [47]. The region between 2200 and 2445 nm was mainly caused by the combined stretching vibrations of O-H, N-H and C-H functional groups, which showed low spectral reflectance [48].
The average spectral curves for mutton, pork and adulterated mixture are shown in Figure 2b. The average spectral curves for different classes of samples demonstrated the same trend but significantly different reflectance. This indicated that differences exist in the chemical composition and structural information between pork and mutton. The average spectral curves for adulterated mixtures of different proportions are presented in Figure 2c. Spectral tendencies of all adulterated samples are basically the same, with some differences in the reflectance. It revealed that different proportions of pork and mutton changed the composition content and material structure of the sample. The results show that the sample spectral information obtained in this study can distinguish amongst pork, mutton and adulterated mixtures, as well as different proportions of pork in mutton. This regularity was obtained under the effect of mutton flavour essence, thereby indicating that the spectral information in this study can avoid the interference of mutton flavour essence.
The original spectra were preprocessed using first derivative combined with multiple scattering correction (1D + MSC). The processed spectra are shown in Figure 2d. The processed average spectral curves of different classes and different proportions are shown in Figure S1 and S2 (Supplementary Materials). Compared with the original spectral curves, preprocessed spectra are more convergent, the influence of random noise is weaker and the reflection and absorption bands are more evident. The subsequent spectral data in this study were all preprocessed with 1D + MSC method.

3.2. Establishment and Evaluation of Classification Model

3.2.1. Model Establishment

BP with three layers of network structure, including input layer, hidden layer and output layer, was used to establish the classification model. Traingdx training function and logsig and purelin transfer functions were selected. The number of hidden layer nodes was traversed in 5 steps from 5 to 100. The traversal results showed that the fitting effect was optimal when the number of hidden layer nodes was 20. ELM was also used to establish the classification model. Sigmoid function was utilised and the hidden layer was traversed in 10 steps from 10 to 200. The fitting was optimal when the number of neurons in the hidden layer was 60. At the same time, support vector machine (SVM) was applied to establish the classification model and grid optimisation was performed to determine c and g. The accuracy of the 10-fold cross-validation was used as the optimisation objective. The accuracy of the classification models in each data set is presented in Table 1. A comparison of the accuracy in the classification models established by BP, ELM and SVM demonstrated that SVM achieves the best modelling results. The accuracy of the calibration and prediction sets was 99.65 and 99.37%, respectively. The performance of SVM was clearly better than that of the BP and ELM models.

3.2.2. Model Optimisation and Comparison

The SSA was used to optimise parameters c and g in the SVM model and was used to optimise the weights and thresholds of the BP and ELM models. The optimisation objective was the maximum accuracy of the 10-fold cross-validation. Optimal parameters were obtained and the models were established when the accuracy, as the fitness values, were the maximum. The results of SSA-BP, SSA-ELM and SSA-SVM are listed in Table 1. The BP, ELM and SVM models optimised by SSA exhibited satisfactory performance, especially the SSA-ELM and SSA-SVM models.
The SSA-BP, SSA-ELM and SSA-SVM models were further compared with the BP, ELM and SVM models. The comparison of the accuracy of each data set before and after the optimisation is presented in Figure 3. The accuracy of each data set of the BP, ELM and SVM models optimised by the SSA was higher than that before optimisation. Amongst them, SSA-ELM showed the highest improvement compared with that before optimisation, followed by SSA-BP and SSA-SVM. The SSA-BP classification model was 0.55% and 0.83% higher than the BP model in the calibration and prediction set, respectively. The performance of the BP model slightly improved after SSA parameter optimisation. The accuracy of SSA-ELM increased by 2.29% and 7.30% in the calibration and prediction set compared with that of the ELM model, respectively. The SSA-ELM model evidently improved. The SSA-SVM model was improved with an accuracy of 99.90% and 99.79% in two data sets. Compared with the SVM model, the SSA-SVM model was 0.25% and 0.42% higher in the calibration set and prediction set. These results showed that the comprehensive classification performance of each model can improve when SSA optimisation is applied and the SSA-SVM model achieves an optimal result.
In order to explore the classification performance of different models for each class of samples, the accuracy of the prediction set is illustrated as a confusion matrix in Figure 4. The single-class accuracy of different models improved after SSA optimisation. Amongst them, the accuracy of the BP model in discriminating mutton (CLASS M) and pork (CLASS P) was low, only 83.33 and 76.67%, respectively. The accuracy of SSA-BP increased to 85 and 81.67% when detecting CLASS M and CLASS P, respectively. Notably, the classification performance still needs to improve, despite showing a slight enhancement after optimisation. The ELM model was insufficient in discriminating CLASS M and CLASS P, with an accuracy of only 43.33 and 88.33%, respectively. The accuracy of SSA-ELM in detecting CLASS M and CLASS P increased to 95 and 93.33%, respectively. Although the improvement was evident, each class contained certain errors. The SVM and SSA-SVM can achieve a detection accuracy of 100% for CLASS M, which was significantly higher than that of other models. The detection accuracy of the SSA-SVM on adulterated mixture (CLASS MP) was the maximum at 100%. The detection accuracy of CLASS P was the highest at 98.33%. These results showed that SSA-SVM achieves the optimal classification performance for each class.
In summary, the performance of the SSA-BP, SSA-ELM and SSA-SVM models improved compared with that of the BP, ELM and SVM models, respectively. Amongst them, the SSA-SVM model obtained the best classification result. Although the mutton flavour essence changed odour information and effected sensory testing, NIR-HSI combined with the SSA-SVM model can effectively identify mutton, pork and adulterated mixtures with pork.

3.3. Establishment and Evaluation of Quantification Model

3.3.1. Model Establishment

The quantification model for the adulterated mixture was established using BP, ELM and SVR. The optimisation objective was the error value of 10-fold cross-validation. The results of the models for predicting pork proportion in an adulterated mixture are listed in Table 2. The SVR model presented the best performance compared with the ELM and BP model. SVR demonstrated an R2 of 0.9290 and 0.8613 and RMSE of 0.0358 and 0.0699 in the calibration set and prediction set, respectively. The RMSE in the prediction set was much higher than that of the calibration set, which was close to twice. Therefore, the models need further optimisation.

3.3.2. Model Optimisation and Comparison

SSA was used to optimise parameters c and g in the SVR model. Meanwhile, SSA was used to optimise the weights and thresholds in the BP and ELM regression models. The optimisation objective was to minimize the error values in 10-fold cross-validation. Optimal parameters were obtained and the SSA-BP, SSA-ELM and SSA-SVR models were established when error values were the minimum. The results of the three models are listed in Table 2. Amongst the models with SSA optimisation, the SSA-SVR model demonstrated the best result with an RC2 and RP2 of 0.9491 and 0.9304, respectively, and an RMSEC and RMSEP of 0.0335 g·g−1 and 0.0458 g·g−1, respectively.
It can be seen from Figure 5 that SSA optimisation for models improves the performance of each data set compared with the use of BP, ELM and SVR models alone. Compared with those of the BP model, the R2 of the calibration set and prediction set of the SSA-BP model for quantification increased by 0.0277 and 0.0778 and RMSE decreased by 0.0053 g·g−1 and 0.0094 g·g−1, respectively. Compared with those of the ELM model, RC2 and RP2 of the SSA-ELM model increased by 0.0340 and 0.0799, whilst RMSEC and RMSEP decreased by 0.0088 g·g−1 and 0.0153 g·g−1, respectively. Compared with those of the SVR model, RC2 and RP2 of the SSA-SVR model increased by 0.0201 and 0.0691, whilst RMSEC and RMSEP decreased by 0.0024 g·g−1 and 0.0241 g·g−1, respectively. These results showed that the quantification performance of all models improves when SSA optimisation is applied and SSA-SVR achieves the optimal quantification performance.
The results in the prediction set of the SSA-SVR with error bar are illustrated in Figure 6. The prediction standard deviation was utilised to reflect the degree of deviation between the predicted value and actual value. It could be seen that all predicted values were above 0% and were distributed near the regression line. This finding showed that all adulteration mixture samples in different proportions were effectively predicted by SSA-SVR [21]. Although the addition of mutton flavour essence changed odour information and effected sensory testing, NIR-HSI combined with the SSA-SVR model can effectively predict the proportion of pork mixed in mutton.

4. Conclusions

In this research, near-infrared hyperspectral imaging combined with machine learning and sparrow search algorithm was used to classify and quantify adulterated mutton with pork under the effect of mutton flavour essence. Classification models using BP, ELM and SVM were developed to discriminate mutton, pork and adulterated mixture samples. Quantification models using BP, ELM and SVR were developed to predict different proportions of pork in mutton. SSA was used to optimise the model parameters and the model results improved after optimisation. The results showed that the optimal classification model is SSA-SVM, with an accuracy of 99.79% in the prediction set, the optimal quantification model is SSA-SVR with an RP2 of 0.9304 and an RMSEP of 0.0458 in the prediction set. The results indicated that the NIR-HSI combined with ML and SSA had the remarkable ability to identify adulterated mutton with pork under the effect of mutton flavour essence. The rapid, non-destructive and automatic method proposed in this study can improve the detection efficiency, reduce the detection cost in practical application and has good application prospects. This study can provide a theoretical and practical reference to the evaluation and supervision of food quality under complex conditions, such as diversified adulteration and multiple food additives.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/foods11152278/s1, Figure S1: Average spectra of different classes preprocessed with 1D + MSC; Figure S2: Average spectra of different proportions preprocessed with 1D + MSC.

Author Contributions

Conceptualization, R.Z.; methodology, B.F.; software, B.F. and D.H.; formal analysis, B.F.; investigation, B.F. and X.C.; resources, R.Z.; data curation, B.F. and S.W.; writing—original draft preparation, B.F.; writing—review and editing, B.F., R.Z. and S.W.; visualization, B.F. and D.H.; supervision, B.F. and R.Z.; validation, B.F. and X.Y.; project administration, R.Z.; funding acquisition, R.Z. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the National Natural Science Foundation of China [Grant No.31860465] and the Bingtuan Innovation Leadership Program in Sciences and Technologies for Young and Middle-aged Scientists [Grant No.2020CB016].

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Mohammed, H.H.H.; Jin, G.F.; Ma, M.H.; Khalifa, I.; Shukat, R.; Elkhedir, A.E.; Zeng, Q.; Noman, A.E. Comparative characterization of proximate nutritional compositions, microbial quality and safety of camel meat in relation to mutton, beef, and chicken. LWT 2020, 118, 108714. [Google Scholar] [CrossRef]
  2. Liu, C.C.; Chu, Z.J.; Weng, S.Z.; Zhu, G.Q.; Han, K.X.; Zhang, Z.X.; Huang, L.S.; Zhu, Z.D.; Zheng, S.G. Fusion of electronic nose and hyperspectral imaging for mutton freshness detection using input-modified convolution neural network. Food Chem. 2022, 385, 132651. [Google Scholar] [CrossRef] [PubMed]
  3. Zhang, Y.Y.; Liu, M.Y.; Wang, S.W.; Kang, C.D.; Zhang, M.Y.; Li, Y.Y. Identification and quantification of fox meat in meat products by liquid chromatography-tandem mass spectrometry. Food Chem. 2022, 372, 131336. [Google Scholar] [CrossRef] [PubMed]
  4. Jackson, L.S. Chemical Food Safety Issues in the United States: Past, Present, and Future. J. Agric. Food Chem. 2009, 57, 8161–8170. [Google Scholar] [CrossRef]
  5. Martins, C.; Rueff, J.; Rodrigues, A.S. Genotoxic alkenylbenzene flavourings, a contribution to risk assessment. Food Chem. Toxicol. 2018, 118, 861–879. [Google Scholar] [CrossRef] [PubMed]
  6. Zheng, M.; Zhang, Y.; Gu, J.; Bai, Z.; Zhu, R. Classification and quantification of minced mutton adulteration with pork using thermal imaging and convolutional neural network. Food Control 2021, 126, 108044. [Google Scholar] [CrossRef]
  7. Wang, Q.; Li, L.; Ding, W.; Zhang, D.; Wang, J.; Reed, K.; Zhang, B. Adulterant identification in mutton by electronic nose and gas chromatography-mass spectrometer. Food Control 2019, 98, 431–438. [Google Scholar] [CrossRef]
  8. Valletta, M.; Ragucci, S.; Landi, N.; Di Maro, A.; Pedone, P.V.; Russo, R.; Chambery, A. Mass spectrometry-based protein and peptide profiling for food frauds, traceability and authenticity assessment. Food Chem. 2021, 365, 130456. [Google Scholar] [CrossRef]
  9. Mandli, J.; El Fatimi, I.; Seddaoui, N.; Amine, A. Enzyme immunoassay (ELISA/immunosensor) for a sensitive detection of pork adulteration in meat. Food Chem. 2018, 255, 380–389. [Google Scholar] [CrossRef]
  10. Wang, F.; Wu, X.F.; Xu, D.S.; Chen, L.P.; Ji, L. Identification of Chicken-Derived Ingredients as Adulterants Using Loop-Mediated Isothermal Amplification. J. Food Prot. 2020, 83, 1175–1180. [Google Scholar] [CrossRef]
  11. Liu, R.; Wang, X.; Wang, X.; Shi, Y.; Shi, C.; Wang, W.; Ma, C. A simple isothermal nucleic acid amplification method for the effective on-site identification for adulteration of pork source in mutton. Food Control 2019, 98, 297–302. [Google Scholar] [CrossRef]
  12. Prachugsorn, A.; Thanakiatkrai, P.; Phooplub, K.; Ouiganon, S.; Sriaead, Y.; Thavarungkul, P.; Kanatharana, P.; Buranachai, C.; Kitpipit, T. Detection of porcine DNA in food using direct asymmetric PCR and catalyzed hairpin assembly fluorescent biosensor: A novel assay for halal food analysis. Food Control 2022, 139, 108989. [Google Scholar] [CrossRef]
  13. Chen, J.R.; Yan, W.J.; Fu, Y.; Wang, L.; Lv, X.Z.; Dai, R.T.; Li, X.M.; Jia, F. The Use of Electronic Nose in the Quality Evaluation and Adulteration Identification of Beijing-You Chicken. Foods 2022, 11, 782. [Google Scholar] [CrossRef] [PubMed]
  14. Tian, X.J.; Wang, J.; Shen, R.Q.; Ma, Z.R.; Li, M.S. Discrimination of pork/chicken adulteration in minced mutton by electronic taste system. Int. J. Food Sci. Technol. 2019, 54, 670–678. [Google Scholar] [CrossRef]
  15. Fengou, L.C.; Lianou, A.; Tsakanikas, P.; Mohareb, F.; Nychas, G.J.E. Detection of Meat Adulteration Using Spectroscopy-Based Sensors. Foods 2021, 10, 861. [Google Scholar] [CrossRef] [PubMed]
  16. Siddiqui, M.A.; Khir, M.H.M.; Witjaksono, G.; Ghumman, A.S.M.; Junaid, M.; Magsi, S.A.; Saboor, A. Multivariate Analysis Coupled with M-SVM Classification for Lard Adulteration Detection in Meat Mixtures of Beef, Lamb, and Chicken Using FTIR Spectroscopy. Foods 2021, 10, 2405. [Google Scholar] [CrossRef]
  17. Gowen, A.A.; O’Donnell, C.P.; Cullen, P.J.; Downey, G.; Frias, J.M. Hyperspectral imaging—An emerging process analytical tool for food quality and safety control. Trends Food Sci. Technol. 2007, 18, 590–598. [Google Scholar] [CrossRef]
  18. Khamsopha, D.; Woranitta, S.; Teerachaichayut, S. Utilizing near infrared hyperspectral imaging for quantitatively predicting adulteration in tapioca starch. Food Control 2021, 123. [Google Scholar] [CrossRef]
  19. Laborde, A.; Puig-Castellvi, F.; Bouveresse, D.J.R.; Eveleigh, L.; Cordella, C.; Jaillais, B. Detection of chocolate powder adulteration with peanut using near-infrared hyperspectral imaging and Multivariate Curve Resolution. Food Control 2021, 119, 107454. [Google Scholar] [CrossRef]
  20. Jiang, H.Z.; Ru, Y.; Chen, Q.; Wang, J.P.; Xu, L.Y. Near-infrared hyperspectral imaging for detection and visualization of offal adulteration in ground pork. Spectrochim. Acta Part A 2021, 249, 119307. [Google Scholar] [CrossRef]
  21. Jiang, H.; Cheng, F.; Shi, M. Rapid Identification and Visualization of Jowl Meat Adulteration in Pork Using Hyperspectral Imaging. Foods 2020, 9, 154. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Leng, T.; Li, F.; Xiong, L.A.; Xiong, Q.; Zhu, M.T.; Chen, Y. Quantitative detection of binary and ternary adulteration of minced beef meat with pork and duck meat by NIR combined with chemometrics. Food Control 2020, 113, 107203. [Google Scholar] [CrossRef]
  23. Nunes, K.M.; Andrade, M.V.O.; Almeida, M.R.; Sena, M.M. A soft discriminant model based on mid-infrared spectra of bovine meat purges to detect economic motivated adulteration by the addition of non-meat ingredients. Food Anal. Method 2020, 13, 1699–1709. [Google Scholar] [CrossRef]
  24. Lu, B.; Han, F.K.; Aheto, J.H.; Rashed, M.M.A.; Pan, Z.G. Artificial bionic taste sensors coupled with chemometrics for rapid detection of beef adulteration. Food Sci. Nutr. 2021, 9, 5220–5228. [Google Scholar] [CrossRef] [PubMed]
  25. Sun, Y.A.; Zhang, M.; Adhikari, B.; Devahastin, S.; Wang, H.X. Double-layer indicator films aided by BP-ANN-enabled freshness detection on packaged meat products. Food Packag. Shelf. 2022, 31, 100808. [Google Scholar] [CrossRef]
  26. Shen, G.H.; Cao, Y.Y.; Yin, X.C.; Dong, F.; Xu, J.H.; Shi, J.R.; Lee, Y.W. Rapid and nondestructive quantification of deoxynivalenol in individual wheat kernels using near-infrared hyperspectral imaging and chemometrics. Food Control 2022, 131, 108420. [Google Scholar] [CrossRef]
  27. Sun, Y.R.; Zhang, H.N.; Liu, G.S.; He, J.G.; Cheng, L.J.; Li, Y.; Pu, F.N.; Wang, H. Quantitative Detection of Myoglobin Content in Tan Mutton During Cold Storage by Near-infrared Hyperspectral Imaging. Food Anal. Method 2022, 15, 2132–2144. [Google Scholar] [CrossRef]
  28. Chen, T.; Ma, L.J.; Tang, Z.Q.; Yu, L.X. Identification of coumarin-based food additives using terahertz spectroscopy combined with manifold learning and improved support vector machine. J. Food Sci. 2022, 87, 1108–1118. [Google Scholar] [CrossRef]
  29. Tang, Y.Q.; Li, C.H. An Online Network Intrusion Detection Model Based on Improved Regularized Extreme Learning Machine. IEEE Access 2021, 9, 94826–94844. [Google Scholar] [CrossRef]
  30. Huang, L.S.; Wu, K.; Huang, W.J.; Dong, Y.Y.; Ma, H.Q.; Liu, Y.; Liu, L.Y. Detection of Fusarium Head Blight in Wheat Ears Using Continuous Wavelet Analysis and PSO-SVM. Agriculture 2021, 11, 998. [Google Scholar] [CrossRef]
  31. Guan, X.; Liu, J.; Huang, Q.R.; Li, J.J. Assessing the Freshness of Meat by Using Quantum-Behaved Particle Swarm Optimization and Support Vector Machine. J. Food Prot. 2013, 76, 1916–1922. [Google Scholar] [CrossRef] [PubMed]
  32. Jiang, X.H.; Xue, H.R.; Zhang, L.N.; Gao, X.J.; Wu, G.D.; Bai, J. Nondestructive detection of chilled mutton freshness based on multi-label information fusion and adaptive BP neural network. Comput. Electron. Agric. 2018, 155, 371–377. [Google Scholar]
  33. Meng, L.; Zhu, R.; Bai, Z.; Zheng, M.; Gu, J.; Ma, B. Discrimination of Chilled Lamb from Different Carcass Parts at Different Storage Times Based on Mobile Phone Images. Food Sci. 2020, 41, 21–26. [Google Scholar]
  34. Xue, J.; Shen, B. A novel swarm intelligence optimization approach: Sparrow search algorithm. Syst. Sci. Control Eng. 2020, 8, 22–34. [Google Scholar] [CrossRef]
  35. Chen, Y.; Duan, W.L.; Yang, Y.; Liu, Z.; Zhang, Y.B.; Liu, J.F.; Li, S.H. Rapid in measurements of brown tide algae cell concentrations using fluorescence spectrometry and generalized regression neural network. Spectrochim. Acta A 2022, 272, 120967. [Google Scholar] [CrossRef]
  36. Zhang, J.N.; Xia, K.W.; He, Z.P.; Yin, Z.X.; Wang, S.J. Semi-Supervised Ensemble Classifier with Improved Sparrow Search Algorithm and Its Application in Pulmonary Nodule Detection. Math. Probl. Eng. 2021, 2021, 6622935. [Google Scholar] [CrossRef]
  37. Gai, J.B.; Zhong, K.Y.; Du, X.J.; Yan, K.; Shen, J.X. Detection of gear fault severity based on parameter-optimized deep belief network using sparrow search algorithm. Measurement 2021, 185, 110079. [Google Scholar] [CrossRef]
  38. Liu, J.P.; Hu, P.W.; Xue, H.R.; Pan, X.; Chen, C. Prediction of milk protein content based on improved sparrow search algorithm and optimized back propagation neural network. Spectrosc. Lett. 2022, 55, 229–239. [Google Scholar] [CrossRef]
  39. Kamruzzaman, M.; ElMasry, G.; Sun, D.W.; Allen, P. Non-destructive assessment of instrumental and sensory tenderness of lamb meat using NIR hyperspectral imaging. Food Chem. 2013, 141, 389–396. [Google Scholar] [CrossRef]
  40. Gao, Q.; Wang, P.; Niu, T.; He, D.J.; Wang, M.L.; Yang, H.J.; Zhao, X.Q. Soluble solid content and firmness index assessment and maturity discrimination of Malus micromalus Makino based on near-infrared hyperspectral imaging. Food Chem. 2022, 370, 131013. [Google Scholar] [CrossRef]
  41. Diwu, P.Y.; Bian, X.H.; Wang, Z.F.; Liu, W. Study on the Selection of Spectral Preprocessing Methods. Spectrosc. Spect. Anal. 2019, 39, 2800–2806. [Google Scholar]
  42. Lu, H.P.; Jiang, H.; Chen, Q.S. Determination of Fatty Acid Content of Rice during Storage Based on Feature Fusion of Olfactory Visualization Sensor Data and Near-Infrared Spectra. Sensors 2021, 21, 3266. [Google Scholar] [CrossRef] [PubMed]
  43. Yan, J.S.; Tian, H.; Wang, S.; Wang, Z.P.; Xu, H.R. Paddy moisture on-line detection based on ensemble preprocessing and modeling for combine harvester. Comput. Electron. Agric. 2022, 198, 107050. [Google Scholar] [CrossRef]
  44. Liu, S.Y.; Wang, S.T.; Hu, C.H.; Zhan, S.J.; Kong, D.M.; Wang, J.Z. Rapid and accurate determination of diesel multiple properties through NIR data analysis assisted by machine learning. Spectrochim. Acta A 2022, 277, 121261. [Google Scholar] [CrossRef]
  45. Yin, H.; Lv, J.W.; Chen, Y.C.; Cen, H.L.; Li, J.B.; Liu, S.Y. Prediction of CO2 Concentration in Xinjiang Breeding Environment of Mutton Sheep Based on LightGBM-SSA-ELM. Trans. Chin. Soc. Agric. Mach. 2022, 53, 261–270. [Google Scholar]
  46. Bai, X.X.; Cao, A.Y.; Cai, W.; Wen, Y.Y.; Liu, Y.Q.; Wang, S.W.; Li, X.W. Rock burst mechanism induced by stress anomaly in roof thickness variation zone: A case study. Geomat. Nat. Hazards Risk 2022, 13, 1805–1830. [Google Scholar] [CrossRef]
  47. Zou, X.B.; Zhao, J.W.; Povey, M.J.W.; Holmes, M.; Mao, H.P. Variables selection methods in near-infrared spectroscopy. Anal. Chim. Acta 2010, 667, 14–32. [Google Scholar]
  48. Pu, H.B.; Kamruzzaman, M.; Sun, D.W. Selection of feature wavelengths for developing multispectral imaging systems for quality, safety and authenticity of muscle foods-a review. Trends Food Sci. Technol. 2015, 45, 86–104. [Google Scholar] [CrossRef]
Foods 11 02278 g001 550
Figure 1. Schematic representation of sample preparation (a), NIR-HSI system (b) and spectral acquisition (c). (A) CCD camera; (B) imaging spectrograph; (C) light source; (D) electric moving stage; (E) sample; (F) light source controller; (G) computer and sample image. ROI: region of interest. NIR-HSI: near-infrared hyperspectral imaging.
Figure 1. Schematic representation of sample preparation (a), NIR-HSI system (b) and spectral acquisition (c). (A) CCD camera; (B) imaging spectrograph; (C) light source; (D) electric moving stage; (E) sample; (F) light source controller; (G) computer and sample image. ROI: region of interest. NIR-HSI: near-infrared hyperspectral imaging.
Foods 11 02278 g001
Foods 11 02278 g002 550
Figure 2. Spectra of samples. (a) Original spectra of all samples; (b) average spectra of different classes; (c) average spectra of different proportions; (d) spectra preprocessed with 1D + MSC.
Figure 2. Spectra of samples. (a) Original spectra of all samples; (b) average spectra of different classes; (c) average spectra of different proportions; (d) spectra preprocessed with 1D + MSC.
Foods 11 02278 g002
Foods 11 02278 g003 550
Figure 3. Comparison for accuracy of models in each data set. (a) Calibration set (10-fold cross-validation); (b) prediction set. BP: back propagation neural network; ELM: extreme learning machine; SVM: support vector machine; SSA: sparrow search algorithm.
Figure 3. Comparison for accuracy of models in each data set. (a) Calibration set (10-fold cross-validation); (b) prediction set. BP: back propagation neural network; ELM: extreme learning machine; SVM: support vector machine; SSA: sparrow search algorithm.
Foods 11 02278 g003
Foods 11 02278 g004 550
Figure 4. Confusion matrix of accuracy for different models in prediction set. (a) BP; (b) SSA-BP; (c) ELM; (d) SSA-ELM; (e) SVM; (f) SSA-SVM. M, P and MP represent mutton, pork, adulterated mixture, respectively.
Figure 4. Confusion matrix of accuracy for different models in prediction set. (a) BP; (b) SSA-BP; (c) ELM; (d) SSA-ELM; (e) SVM; (f) SSA-SVM. M, P and MP represent mutton, pork, adulterated mixture, respectively.
Foods 11 02278 g004
Foods 11 02278 g005 550
Figure 5. Comparison of R2 and RMSE for different models in each dataset. (a,b) Calibration set (10-fold cross-validation); (c,d) prediction set. RC2: coefficient of determination in calibration set; RP2: coefficient of determination in prediction set; RMSEC: root mean squared error for calibration set; RMSEP: root mean squared error for prediction set.
Figure 5. Comparison of R2 and RMSE for different models in each dataset. (a,b) Calibration set (10-fold cross-validation); (c,d) prediction set. RC2: coefficient of determination in calibration set; RP2: coefficient of determination in prediction set; RMSEC: root mean squared error for calibration set; RMSEP: root mean squared error for prediction set.
Foods 11 02278 g005
Foods 11 02278 g006 550
Figure 6. The result in prediction set of the SSA-SVR with error bar.
Figure 6. The result in prediction set of the SSA-SVR with error bar.
Foods 11 02278 g006
Table
Table 1. The discrimination results in each data set using the basic models and the optimisation models.
Table 1. The discrimination results in each data set using the basic models and the optimisation models.
ModelsCalibration Set
(10-Fold Cross-Validation)		Prediction Set
Accuracy (%)Accuracy (%)
BP92.92%91.67%
ELM97.15%90.83%
SVM99.65%99.37%
SSA-BP93.47%92.50%
SSA-ELM99.44%98.13%
SSA-SVM99.90%99.79%
Note: BP: back propagation neural network; ELM: extreme learning machine; SVM: support vector machine; SSA: sparrow search algorithm.
Table
Table 2. The prediction results in each data set using the basic models and the optimisation models.
Table 2. The prediction results in each data set using the basic models and the optimisation models.
ModelsCalibration Set
(10-Fold Cross-Validation)		Prediction Set
RC2RMSEC (g·g−1)RP2RMSEP (g·g−1)
BP0.84640.06360.72830.0833
ELM0.89100.05420.79460.0739
SVR0.92900.03590.86130.0699
SSA-BP0.87410.05830.80610.0739
SSA-ELM0.92490.04540.87450.0586
SSA-SVR0.94910.03350.93040.0458
Note: RC2: coefficient of determination in calibration set; RP2: coefficient of determination in prediction set; RMSEC: root mean squared error for calibration set; RMSEP: root mean squared error for prediction set; SVR: support vector regression.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Fan, B.; Zhu, R.; He, D.; Wang, S.; Cui, X.; Yao, X. Evaluation of Mutton Adulteration under the Effect of Mutton Flavour Essence Using Hyperspectral Imaging Combined with Machine Learning and Sparrow Search Algorithm. Foods 2022, 11, 2278. https://doi.org/10.3390/foods11152278

AMA Style

Fan B, Zhu R, He D, Wang S, Cui X, Yao X. Evaluation of Mutton Adulteration under the Effect of Mutton Flavour Essence Using Hyperspectral Imaging Combined with Machine Learning and Sparrow Search Algorithm. Foods. 2022; 11(15):2278. https://doi.org/10.3390/foods11152278

Chicago/Turabian Style

Fan, Binbin, Rongguang Zhu, Dongyu He, Shichang Wang, Xiaomin Cui, and Xuedong Yao. 2022. "Evaluation of Mutton Adulteration under the Effect of Mutton Flavour Essence Using Hyperspectral Imaging Combined with Machine Learning and Sparrow Search Algorithm" Foods 11, no. 15: 2278. https://doi.org/10.3390/foods11152278

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop