THz Data Analysis and Self-Organizing Map (SOM) for the Quality Assessment of Hazelnuts

Abstract

In recent years, the use of techniques based on electromagnetic radiation as an investigative tool in the agri-food industry has grown considerably, and between them, the application of imaging and THz spectroscopy has gained significance in the field of food quality control. This study presents the development of an experimental setup operating in transmission mode within the frequency range of 18 to 40 GHz, which was specifically designed for assessing various quality parameters of hazelnuts. The THz measurements were conducted to distinguish between healthy and rotten hazelnut samples. Two different data analysis techniques were employed and compared: a traditional approach based on data matrix manipulation and curve fitting for parameter extrapolation, and the utilization of a Self-Organizing Map (SOM), for which we use a neural network commonly known as the Kohonen neural network, which is recognized for its efficacy in analyzing THz measurement data. The classification of hazelnuts based on their quality was performed using these techniques. The results obtained from the comparative analysis of coding efforts, analysis times, and outcomes shed light on the potential applications of each method. The findings demonstrate that THz spectroscopy is an effective technique for quality assessment in hazelnuts, and this research serves to clarify the suitability of each analysis technique.

1. Introduction

Hazelnuts in countries such as Italy and Turkey, the world’s leading producer, play an important role in the agri-food industry due to their various nutritional properties. In order to assess the final quality of the hazelnut, several controls are implemented to monitor the main parameters, such as geographical origin, harvest time, and morphological and chemical characteristics [1,2,3].

One of the main defects affecting product quality and hence market value is rotten hazelnuts, which is often associated with a bitter taste. Furthermore, under certain conditions, the rotten fruit may be covered with a white patina of mold [4]. One of the most important request from the hazelnut industry is to find more efficient methods for the identification of rotten fruit, which can ruin the quality of entire batches of hazelnuts if put into production.

In last few decades, there has been a great increase in requests by the agri-food industry to use technologies dedicated to food quality control. In particular, the use of diagnostic technologies, both to inspect food, but above all to improve its quality, has grown considerably. A great interest is therefore directed to those technologies that are based on electromagnetic radiation (non-invasive) for the sorting of products and the detection of microorganisms, such as molds. Today, several techniques are used for food quality control. These include Raman spectroscopy [5], X-ray devices [6,7,8], thermal infrared [9,10] and finally microwave-based instruments [11,12,13].

Of course, all these techniques have their specific features and limitations; for example, Raman spectroscopy, together with imaging techniques, is able to provide information on the chemical composition of the material, but at the same time, they are expensive methods; X-ray techniques in the food industry allow the acquisition of high-resolution images, but present risks related to the use of this radiation which, being ionizing and invasive, require special precautions to protect both the operator and the product being examined. In contrast, infrared (IR)-based technologies have the great advantage of being safe for the operator, but at the same time, they have a limited ability to penetrate matter. One example is IR spectroscopy, which has been used to check the quality of shelled hazelnuts in terms of moisture and acidity [14], defective kernels (surface mold and dark skin color) and lipid oxidation [15]. With regard to the detection of bugged hazelnuts, RGB (Red–Green–Blue) image analysis was found to be reliable for identifying damage on halved hazelnuts [16]. However, the possibility to sort the hazelnut when still intact is fundamental for the food industry. Furthermore, the RGB method, being a surface analysis, cannot detect hidden defects and therefore cannot guarantee a complete selection of the product.

More recently, emerging techniques operating in the terahertz band, such as imaging and THz spectroscopy, due to their characteristics, are becoming important in this context. These techniques have been applied in various sectors, including the biomedical field [17], precision agriculture [18], food quality control [19,20], cultural heritage [21,22] and aerospace industry [23,24]. Finally, techniques based on terahertz spectroscopy have been applied for the detection of hazardous mycotoxins, including aflatoxins, in agricultural food products [25], such as almonds. However, this device is difficult to use in practice due to the numerous parameters that need to be monitored. One example is the temperature and humidity in the environment, which at these frequencies influence the THz signal. An interesting property of terahertz radiation concerns its non-ionizing, non-destructive and non-invasive nature. This allows both the operator not to be subjected to risks and the operator not to trigger chemical changes in the food. Thanks to its longer wavelength, THz radiation undergoes a lower Rayleigh scattering than infrared radiation, so it can result in a greater depth penetration. Since terahertz radiation excites rotational and roto/vibrational transitions, polar liquids such as water are sensitive to these frequencies, exhibiting both strong absorption and reflection. So, while this allows us to determine the amount of moisture in food or water in biological tissues, including leaves and plants, it also severely limits deep penetration [26].

Some chemical compounds have an absorption spectrum in the THz region [27]; this detail leads to the detection of all those chemical substances present in foods that show a significant fingerprint in the THz band. Moreover, at these frequencies, dielectric materials such as plastic and paper are transparent whereby the contents of a package can be easily analyzed using this radiation [28].

Moreover, it worth noting that in recent years, interest in neural networks applied to the field of agri-food field has grown.

Indeed, artificial neural networks (ANNs) can be used to inspect the quality of food products [29,30,31,32,33,34,35,36,37], like wine and beer [38,39,40,41,42,43,44,45,46,47,48]. In an artificial neural network, the learning process can be either supervised or unsupervised. In the first case, supervised learning is trained to produce desired outputs in response to sampled inputs. In unsupervised networks, the search for a logical classification of input data is left to the network itself.

This is the case of self-organizing maps (SOMs), whose most common SOM model is the one proposed by the Finnish researcher Teuvo Kohonen in 1982 [49,50]. The main fields of interest of SOMs include pattern recognition, robotics, telecommunications and medical applications [51,52,53]. This type of neural network is widely used as a clustering tool by detecting the most representative patterns in the dataset [54].

In this study, millimeter wave measurements to select healthy, empty and rotten hazelnuts samples have been made by using an experimental setup designed to perform measures in transmission configuration. Two different data analysis techniques have been used for the sorting of hazelnut samples: one relied on a statistical-based algorithm, and another one is more complex and based on the use of an unsupervised neural network.

2. Experimental Setup

This measurement system has been designed to measure the transmission of samples in the frequency range between 18 and 40 GHz. An outline of the experimental setup is shown in Figure 1. Millimeter wave radiation emitted from a Yttrium–Iron Garnet (YIG) tunable source is sent, using a transmission line, to a horn antenna, which launches it into free space. The radiation then passes through the sample and, after being collected by another horn antenna, is detected by a Schottky diode, which gives a voltage signal proportional to the transmitted radiation power. The voltage signal is measured by a digital multimeter.

Since the electromagnetic radiation at the edge of the horn antenna is partially reflected, both in launching and collecting, due to the impedance mismatch between the antenna and free space, the optical system, throwing horn–sample–collection horn, can be seen as a system composed of three partially reflecting mirrors. The intensity of the transmitted radiation therefore depends not only on the absorption or scattering of the sample but also on the transfer function of the composite system of the three Fabry–Pérot, which depends clearly on the frequency in use. As a function of frequency, we will have a transmitted oscillating signal similar to the Etalon’s transfer function eventually modified by the presence of the sample. Thus, a comparison of the transmitted signals as a function of frequency with and without the sample provides information about the optical properties of the studied sample. Figure 1A shows an image of the system with its components, while Figure 1B shows an image of the layout.

The millimeter wave source is a YIG oscillator MLOS-184 (Micro Lambda Wireless, Fremont, CA, USA) operating in the region between 18 and 40 GHz with a maximum power of about 20 mW. The detector used in this system is a Schottky diode, which is capable of detecting frequencies in the spectral region emitted by the source.

3. Samples

In Italy, there are two varieties of IGP (Protected Geographical Indication) branded hazelnuts: the so-called “Tonda delle Langhe” (from Piedmont) and the “Tonda di Giffoni” (from Campania) and one, with a DOP (Protected Designation of Origin) label, the “Tonda Gentile Romana”.

In this study, the samples under examination belong to the Tonda Gentile Romana and San Giovanni varieties, which are typical cultivars of the Lazio and Campania regions. The objective of this research is to select hazelnuts without breaking the outer shell, so millimeter wave measurements were carried out on apparently healthy hazelnuts and rotten hazelnuts, as shown in Figure 2. A parameter to be considered is the amount of moisture present in hazelnuts in order to ensure proper storage of the product. In fact, during roasting, there is a decrease in moisture from 4–6% to 1–3%.

Transmission measurements were performed on 212 random hazelnuts containing either healthy, rotten, shriveled and empty hazelnuts. A subset of fifty hazelnuts was used to train the neural network and allow automatic sorting by a self-organizing neural network using the Kohonen algorithm

4. Results

Data are analyzed using a simple algorithm and the neural network to sort out the empty or rotten hazelnut, as explained in detail in the following paragraphs. To carry out the measurements, each hazelnut sample was placed inside a petri dish and surrounded by a layer of eccosorb in order to focus the radiation only on the hazelnut. The frequencies used are between 36 and 40 GHz: nine frequencies corresponding to the points shown in Figure 3 were used (36 GHz, 36.64 GHz, 37.21 GHz, 37.77 GHz, 38.34 GHz, 38.90 GHz, 39.47 GHz, 40.03 GHz, 40.60 GHz).

Figure 3 shows the trend over frequency of the transmission values measured for ten hazelnut samples. A possible sorting based on the transmission values between healthy and rotten hazelnut seems possible for a small dataset and for single measurements. This will not be any more possible on bigger datasets. This difference can be explained by the fact that at these frequencies, the water and fatty acids present inside the hazelnut absorb and reflect the radiation, showing lower transmission values.

4.1. Statistical Data Analysis Methods

As a tested in the analysis of the hazelnut condition, a simple algorithm was implemented to distinguish between ‘healthy’ and ‘rotten’ or defective hazelnuts. The available data consist of measurements taken with the THz instrumentation described in paragraph 1. The experimental dataset consists of 9 points per sample given by measurements at frequencies between 36 and 40 GHz. The samples are those described in paragraph 2, which are appropriately classified ex post. The algorithm, implemented using the Python 3 language, was developed and then verified on the available dataset.

To gain an understanding and enable a comparison with data analysis using a neural network, as will be seen in the next paragraph, details of the steps taken to achieve the final selection result are provided here. The aim is to compare not only the results achieved but also the ‘effort’ required to achieve them. In this approach, the observation of experimental data is fundamental in determining the identifying parameters of the hazelnut’s ‘condition’. Therefore, it starts with the representation of the collected experimental data. The raw trend of the values obtained in the ‘transmission’ configuration (volts) acquired in the specified frequency range (36–40 GHz) does not seem, as initially hypothesized, to allow for the definition of a threshold value to discriminate between samples. This is also due to the measurement configuration and the variable sizes of the samples. In Figure 4, the trend of the measured voltage value in transmission for various samples as a function of frequency is shown. The curve with the highest values is related to the ‘petri-dish’, i.e., the measurement without the sample. The curves associated with various samples are represented with two different colors: red for samples classified ex-post as ‘bad’ and blue for ‘good’ ones.

Although theoretically, it may seem that one can easily distinguish based on the measured intensity, the complexity and variability of hazelnut geometries, combined with the measurement configuration, make this differentiation impossible to achieve ‘as is’.

However, if we average the signals obtained at various frequencies and represent hazelnuts as ‘good’ and ‘bad’ using the same colors as in Figure 5, it appears that average transmission values can be determined to allow discrimination between different hazelnuts. It seems possible to establish threshold values below which hazelnuts can be considered ‘good’ and above which they can be considered ‘bad’.

Assuming the primary goal is to eliminate the bad hazelnuts that can affect the quality of an entire batch, attention will be focused on detecting bad hazelnuts to remove them from the general set of hazelnuts. The analysis will be based on the percentage of hazelnuts classified as ‘bad’ compared to the number of hazelnuts that were actually verified to be ‘bad’ and the number of ‘good’ hazelnuts that would be ‘sacrificed’ under these criteria—in other words, the percentage of good hazelnuts that would be classified as ‘bad’.

The full procedure developed for hazelnut classification consists of two steps: one based on the average value at the measured frequencies (A), Figure 5, and a second step based on the verification of the presence of a peak (maximum) at the frequency of 36.46 GHz in the trends over frequency. In the measurement configuration used, a difference between ‘good’ and ‘bad’ is indeed highlighted at this frequency, particularly observing that ‘good’ hazelnuts have a maximum at this frequency.

The double condition imposed on the data appears to have an excellent percentage in hazelnut selection. Let us analyze the results obtained on a sample of 212 hazelnuts, including 140 without defects and 72 with some form of defect not visible from the outside (rotten, shriveled, or empty). Taking the threshold value that is evident in Figure 5 (blue), but even more so in Figure 3, as s = 55 mV, and applying the algorithm, the following results are obtained in the classification:

• Number of hazelnuts classified as good: 140, including one bad → sorting error: 0.7%.
• Number of hazelnuts classified as bad: 70, including one good → sorting error: 1.4%.

In general, an error of about 2 hazelnuts out of 212 is obtained, which is approximately a 1% error in hazelnut identification. Another ‘limit’ threshold value could be one where 100% of the ‘bad’ hazelnuts are to be eliminated. To achieve this, we can consider a threshold value of 25 mV. With this threshold value, the algorithm returns:

• Number of hazelnuts classified as bad: 79, including 9 good → sorting error: 0%.
• Percentage of good hazelnuts lost: 11%.

4.2. Neural Networks Methods

Using MATLAB software (R2023a) it was possible to classify the data set consisting of 212 hazelnuts by means of a SOM (self-organizing map). The use of an unsupervised neural network has been investigated for sorting hazelnuts. Such a neural network consists of two layers of neurons, the input layer and the output layer. Specifically, our input is given by the nine transmission values, corresponding to the nine frequencies acquired for each hazelnut. A simplified image of the learning process is below (see Figure 6).

Once the nine transmission values for each hazelnut have been acquired, they were used to feed the input layer (see Figure 6). Whereas in nature, communication between neurons takes place via synapses, in a neural network, this takes place via the so-called input weights of neurons. Looking closely at Figure 6, it is clear how each input neuron is connected to each output neuron via these weight vectors. The weight vectors in this case are a numerical value associated with a specific connection among neurons, representing the strength (importance) of that connection.

In our case, the learning algorithm for the selection of hazelnuts through the creation of clusters follows three processes: a competitive process, a cooperative process and an adaptation process. In the competitive process, the neurons present in the output layer compete with each other to occupy specific areas of the map by configuring clusters in the best possible way. To do this, the nine transmission values associated with each hazelnut sample presented to the network are compared with the weight vector of each neuron present in the output. The output neuron that has the “comparable” weight vector to the input sample will be declared the winner of the competition (best matching unit, BMU). One of the most frequently used functions to determine the winning neuron is the Euclidean distance, as shown in the following formula:

$${\mathrm{d}}_{\mathrm{i}}(\mathrm{t})=||\mathrm{x}(\mathrm{t})-{\mathrm{w}}_{\mathrm{i}}(\mathrm{t})||=\sqrt{{\sum }_{\mathrm{j}=1}^{\mathrm{m}}({\mathrm{x}}_{\mathrm{t}\mathrm{j}}-{\mathrm{w}}_{\mathrm{t}\mathrm{j}\mathrm{i}}{)}^2}\mathrm{i}=1,2,3\dots \mathrm{n}$$

(1)

where (x) represents the input data, and (w_i) represents the weight vectors at iteration (t).

Once the Euclidean distance between the input neurons and all the weight vectors of the output neurons has been calculated, the winning neuron is the one with the lowest value:

c(t) = arg min{||x(t) − w_i (t)||}

(2)

During the cooperative process, the winning neuron can determine the spatial position within the output space; neighboring neurons can be influenced by the winning neuron. Finally, in the adaptation process, the winning neuron and its neighbors in the output space can update the weight vectors through the so-called weight adjustment process to emulate the characteristics of the input space. This process is repeated each time a new stimulus, and thus a new hazelnut, is presented to the network, resulting in a change in the weight vectors. The following formula is used to adjust the weights at each iteration:

w_i (t +1) = w_i (t) +α(t)[x(t) − w_i(t)]

(3)

where w_i(t) is the weight vector of the winning neuron at iteration t, while w_i (t + 1) is the updated weight vector at the next iteration (t + 1), and finally, α(t) is the learning rate. This last parameter controls the rate of change in the weight vectors (takes values between 0 and 1) and gradually decreases with increasing epochs or iterations (t).

For this first test, a 2-by-2 grid with a number of neurons equal to four was used. Figure 7a shows a neural map with a hexagonal topology; specifically, the map highlights the positions of the neurons within the network, indicating how many observations are associated with each neuron. From the image, each neuron shows a number indicating the amounts of hazelnuts classified according to certain characteristics. In Figure 7b, the SOM Neighbor Weight Distances is shown; it precisely indicates the distances between neighboring neurons. The blue hexagons represent the neurons, while the colors visible in the map indicate the distances between neurons; darker colors represent larger distances, while lighter colors represent smaller distances.

From Figure 8, it can be seen that the neural network has created a non-homogeneous group of neurons highlighted by the dark colors, in which rotten, shriveled and empty nuts fall. In contrast, the healthy hazelnuts fall in the yellow cluster.

Table 1. 70% of the hazelnuts were healthy, 10% were rotten, 6% were shriveled, and finally 14% were empty.

Neuron Index	Hazelnuts in Each Neuron	Only Healthy	Only Rotten	Only Shriveled	Only Empty	% Healthy	% Rotten	% Shriveled	% Empty
1	6	0	4	0	2	70%	10%	6%	14%
2	17	17	0	0	0
3	10	1	1	3	5
4	17	17	0	0	0
tot	50	35	5	3	7

provides a summary of the results.

In this first test performed on fifty random hazelnuts, used as a neural network training test, it was seen how the unsupervised neural network was able to create two different clusters, one homogeneous and the other non-homogeneous. Most of the healthy hazelnuts fell into the homogeneous cluster. The following section will show the results of the largest data set.

For the selection of one hundred and sixty-two hazelnuts, the neural network was first trained with a data set consisting of fifty hazelnuts, as reported above, in order to create a selection method. In the first step, the network was trained with fifty hazelnuts, while in the second and final step, the method was validated using a larger data set of one hundred and sixty-two hazelnuts. A map of the distribution of the second data set is shown below in Figure 9.

From Figure 9, it can be seen that the rotten, shriveled and empty hazelnuts fall into the so-called bad neurons (black neurons), which are the same ones used by the network during the training process. The healthy hazelnuts, on the other hand, fall into the good neurons (yellow neurons), which are again the same ones used in the training process. Thus, in the black-colored neurons fall rotten, shriveled and empty hazelnuts, while in the yellow-colored neurons, only the healthy ones fall.

Table 2. 65.43% of the hazelnuts were healthy, 12.34% were rotten, 5.55% were shriveled and finally, 16.66% were empty with a sorting error of 0.6%.

Neuron Index	Hazelnuts in Each Neuron	Only Healthy	Only Rotten	Only Shriveled	Only Empty	% Healthy	% Rotten	% Shriveled	% Empty	Sorting Error
1	28	0	12	4	12	65.43%	12.34%	5.55%	16.66%	0.6%
2	105	105	0	0	0
3	27	0	7	5	15
4	2	1	1	0	0
tot	162	106	20	9	27

provides a summary of the results.

4.3. Discussion

The case study has a rather limited number of parameters, allowing for the development of a simple sorting algorithm based on statistical methods. This algorithm is compared with a more advanced and complex method based on neural networks. As highlighted in the previously paragraphs, both methods are capable of distinguishing between healthy and unhealthy hazelnuts. The simplicity of the algorithm, referred to as ‘classic’, and its relatively low computational effort are evident. However, this simplicity comes at the cost of a system with less adaptability and greater susceptibility to variations in the measurement system. Calibration is required for each change in the measurement configuration.

On the other hand, the sorting system based on the use of neural networks is more flexible and has greater potential. In this case, nine transmission values acquired for each hazelnut represent the input of the network. For this initial test, a 2-by-2 grid with a total of four neurons was employed. The neural network successfully formed a homogeneous cluster, encompassing most of the healthy hazelnuts, and a non-homogeneous cluster (highlighted by darker colors), associated with rotten, shriveled, and empty hazelnuts. The increased development effort and computational complexity are undoubtedly outweighed in the long run by the flexibility and potential of the method.

For example, the ability to distinguish between different types of unhealthy hazelnuts and the ability to search for specific types of defects, particularly those within the clusters, makes the system attractive for more in-depth studies and developments.

If a more rudimentary and cost-effective system is desired, capable of simply selecting the ‘good’ hazelnuts or eliminating the ‘bad’ ones, the system based on the traditional algorithm may still be of interest.

5. Conclusions

In this study, millimeter wave measurements to select healthy, rotten, shriveled and empty hazelnuts samples have been made by using an experimental setup designed to perform measures in transmission configuration. Measurements were carried out by selecting nine frequency values between 36 and 40 GHz. Two different methods are compared to analyze the same data: one based on some trivial statistics and another based on neural networks.

Both systems can distinguish between healthy and unhealthy hazelnuts with sufficient accuracy. The two systems have different development costs in terms of computational capabilities and development. These costs, higher in the case of neural networks, are widely compensated by the enormous possibilities for further development and greater depth of selection capabilities (such as differentiation between different types of defects). The system based on the statistical algorithm may be cost-effective only for systems aimed at a coarser sorting and with a specific task: eliminating all the bad ones or selecting all the good ones.

The method based on neural networks (the so-called Self-Organizing Map, which exploits Kohonen’s algorithm) allows the achievement of both objectives and has potential still under study but at a higher implementation cost. The proposed methods have both proven to be able to discriminate with high accuracy, showing different features.

Further studies are currently underway in order to test the neural network on a larger data set, also trying to improve the performance of the setup by reducing acquisition times through the implementation of control software.