Next Article in Journal
CorDeep and the Sacrobosco Dataset: Detection of Visual Elements in Historical Documents
Previous Article in Journal
Hair Follicle Classification and Hair Loss Severity Estimation Using Mask R-CNN
Previous Article in Special Issue
Pore Segmentation Techniques for Low-Resolution Data: Application to the Neutron Tomography Data of Cement Materials
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
J. Imaging
Volume 8
Issue 10
10.3390/jimaging8100284
jimaging-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Neutron Imaging and Learning Algorithms: New Perspectives in Cultural Heritage Applications

by
Claudia Scatigno
* and
Giulia Festa
*
CREF—Museo Storico della Fisica e Centro Studi e Ricerche Enrico Fermi, Via Panisperna 89a, 00189 Rome, Italy
*
Authors to whom correspondence should be addressed.
J. Imaging 2022, 8(10), 284; https://doi.org/10.3390/jimaging8100284
Submission received: 11 July 2022 / Revised: 4 October 2022 / Accepted: 12 October 2022 / Published: 14 October 2022
(This article belongs to the Special Issue Computational Methods for Neutron Imaging)
Browse Figures
Versions Notes

Abstract

:
Recently, learning algorithms such as Convolutional Neural Networks have been successfully applied in different stages of data processing from the acquisition to the data analysis in the imaging context. The aim of these algorithms is the dimensionality of data reduction and the computational effort, to find benchmarks and extract features, to improve the resolution, and reproducibility performances of the imaging data. Currently, no Neutron Imaging combined with learning algorithms was applied on cultural heritage domain, but future applications could help to solve challenges of this research field. Here, a review of pioneering works to exploit the use of Machine Learning and Deep Learning models applied to X-ray imaging and Neutron Imaging data processing is reported, spanning from biomedicine, microbiology, and materials science to give new perspectives on future cultural heritage applications.

1. State-of-the-Art in Imaging and in Machine and Deep Learning

1.1. Imaging Techniques

Imaging is a powerful tool for the presentation of multi-dimensional and multi-parameter data. After data acquisition and processing, volume and surface rendering can be employed for data visualization and quantitative analyses. Imaging is used in various applications, such as engineering, remote sensing, medicine, forensic studies, and materials science [1]. The acquired dataset can be pre-processed for optimization purposes as a function of the specific setup and scope of the study. Imaging techniques produce a large amount of data, and the data analysis requires a large computational effort. Moreover, to obtain the best quality of the data, the image noise reduction is an important factor. Usually, different approaches are applied according to the characteristics of the specific sensors and the setup used for the acquisition process. Signal-to-noise ratio and resolution improvement are recently assessed through classes of Machine Learning techniques, such as Deep Learning [2].

1.2. Neutron Imaging and Challenges

Neutron Imaging (NI), such as radiography and tomography, are testing methods which can be used to determine the inner structure of the investigated objects. The study of the morphology and internal volumes is useful for the determination of the manufacturing processes, provenance, dating, and information about the state of conservation and restoration in the cultural heritage field [3,4,5,6]. The NI techniques exploit the interaction of the incoming neutron beam with the object under investigation. The neutron beam is registered by a position-sensitive detector after the interaction with the object. The detector system registers the information pixel-wise, while the intensity per pixel is separated into grey values, commonly in a 16-bit format, enabling to distinguish more than 65,000 intensity variations and converting into grey-scale values. This digital format allows for more advanced analysis methods such as tomography, real-time imaging, energy-selective and diffractive imaging, and grating interferometry [7,8,9]. These methods convert radiographic data into compressed information with the help of dedicated software tools. Due to the interaction processes of neutrons with the matter, they are considered an ideal probe for the non-destructive and non-invasive investigation of cultural heritage objects [3,4,5,6,7,8,9,10,11,12,13]. Indeed, neutrons interact with the atomic nuclei with a penetration depth of incident beam that is a function of the sample: for metal alloys, pottery, and stones they access the bulk of the objects without substantial attenuation while they are highly sensitive to light elements such as hydrogen [3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18]. Significant progress has been made regarding spatial and temporal resolutions, achieved in different ways such as the optimization of the optical camera that collects the signal from the scintillator screen [19] and the use of axisymmetric grazing-incidence focusing mirrors [20,21], transforming pinhole cameras into microscopes. In the framework of cultural heritage, an important issue to consider is the activation of the irradiated samples and the deactivation time that is related to the lifetime of the compound nuclei, particularly for metallic objects [3]. Indeed, the ancient objects preserved at museums must be returned within a short period with deadlines already defined in the experimental design step, and the activation parameters need to be under control. For this reason, a balance between the acquisition time has to be taken during the data acquisition phase: long acquisitions guarantee a good image quality but could dramatically activate the samples. When a sample is irradiated by a neutron beam, nuclear reactions occur such as neutron absorption, and a compound nucleus is formed producing an induced activity that is a function of the kind of isotopes and their abundance in the sample, the neutron capture cross-sections, and isotopes half-life. The latter needs to be in the order of seconds to minutes for short-lived isotopes, otherwise, the samples risk remaining active for a long time, such as in the case of the antimony (Sb122 and Sb124) with a half-life of the order of months. To avoid these activation problems, preliminary measurements via portable techniques such as X-ray fluorescence or infrared spectroscopies are carried out before moving the objects at the neutron facilities to have an idea about the composition for the prediction of the deactivation time after the neutron irradiation. These portable techniques give us only surface information and for non-homogeneous objects, eventual activation due to materials inside the ancient object, such as a sealed ceramic vase, is not predictable. For this reason, as a first step of the NI experiments, some short radiographies to check the activation of the sample are generally carried out before starting the complete tomographic scan.

1.3. Neutron Imaging Optimization

To obtain good NI data, quality improvements to the radiographic images are developed in previous works thanks to specific algorithms to find the best possible settings to the detriment of phase sensitivity and spatial resolution [22]. The data acquisition could be optimized through experimental procedures related to the beam characteristics and collimation, optical and detection systems, and secondary effects such as multiple scattered neutrons, and γ radiation background at the beamlines [23,24,25]. Indeed, the neutron source characteristics, such as the neutron fluence rate, could have statistical fluctuations in terms of time and space, which gives rise to the Gaussian noise, or mixed Poisson–Gaussian noise, which can affect the image quality [23]. In the data analysis process, one of the challenges in cultural heritage applications is the segmentation of the internal regions because the objects are generally composed of different materials where the edges are not sharp due to corrosion and degradation processes, or the penetration of one phase in another is very common, such as the case of the liquid vase content in the ceramic or stone matrix (Figure 1).

1.4. Neutron Imaging and Deep Learning

Recently, Deep Learning techniques are applied in the NI context to address the sensitivity-resolution trade problem [23]. The latter is based on a set of algorithms that attempt to model high-level abstractions in data by using model architectures, with unsupervised or semi-supervised feature learning and hierarchical feature extractions [26]. Among them, the neural network-based prediction methods, such as Convolutional Neural Networks (CNNs), are recent and are able to reduce the data acquisition time without compromising the resolution of the structure characterization to improve the temporal resolution of the time-dependent measurement [26,27]. CNNs are particularly suitable for data processing and data analysis, i.e., for object recognition, object detection, scene recognition, semantic segmentation [28,29,30], action recognition, object tracking, and many other tasks. Furthermore, Spiking Neural Networks (SNNs), are deployed on neuromorphic computing hardware which demonstrates an ultra-low power consumption [31,32]. The main purpose of Machine Learning (ML) is the reduction of high-dimensional data and can be categorized into two classes: data compression and data visualization. The supervised ML, such as the K-Nearest Neighbour (KNN) algorithm and Support Vector Regression (SVR), can reduce data and compare the training data with the predicted data by regression models, in order to attribute and validate unlabelled datasets. These techniques could be used as a selective method to extract the more significant data useful to reconstruct the image, i.e., selecting the data identified as Singular Vector (SV) belonging to the class of separation. These techniques are successfully applied in other fields [33,34] and could be employed in the cultural heritage framework. They are used to denoise the reconstruction images [33], improve their quality [33], and for optimization processes [35].
Here, a review of pioneering works to exploit the use of learning models applied to Neutron Imaging data processing is reported. The present work aims to provide an overview of the Machine Learning and Deep Learning approaches used at different levels of data imaging analysis and in different sectors that could be useful in the cultural heritage domain. Due to the similarities between X-rays, magnetic resonance imaging data, and Neutron Imaging, some applications are also reported because they can be easily translated to the neutron field.

2. Deep Learning Applications

Deep Learning (DL) models are successfully applied in different stages of data processing from the acquisition to the data analysis in several contexts and with different investigation techniques, spanning from X-ray computed tomography (XCT) to Neutron Imaging [23,26,33,34,35,36,37,38,39,40,41].

2.1. Learning Algorithms in X-ray Tomography (Biomedicine, Materials Science, and Cultural Heritage)

CNN algorithms are DL models widely used for the analysis of visual imagery, eliminating overfitting by Kernel function. They use the neural networks to detect features of images by a hierarchical structure of the data and assemble simple patterns to generate complex patterns for regularization. Amongst widespread imaging techniques, X-ray tomography is used to investigate the internal structures of a large variety of opaque samples in a non-destructive and non-invasive way. Recently, Spiking Neural Networks (SNNs) [40] are also successfully applied in medicine on magnetic resonance imaging and X-ray tomography datasets with the potential to reduce power consumption while maintaining a good performance via neuromorphic computing hardware.
The most recent applications of DL techniques exploit biomedical X-ray dataset images to detect early diagnosis [42,43,44], estimate some specific properties of complex materials [36], and characterize ancient woodblocks [45], as outlined in Table 1.

2.2. Learning Algorithms in Cultural Heritage Imaging

Machine Learning and Deep Learning algorithms are successfully applied to cultural heritage. In Table 2, a summary of a selection of learning methods is listed to present the recent scenarios in this field [46,47,48,49,50]. In detail: (1) ML-SVM is a supervised Machine Learning algorithm that separates data points by a hyperplane and is successfully used in handwriting recognition; (2) ML-RF is a random forest that produces outputs that are collected and combined as a tree, useful in classification and segmentation; (3) DL-CNN is a Convolutional Neural Network very suitable for image classification.
To the best of our knowledge, DL has not yet been applied extensively in NI for cultural heritage. Rather, DL was developed and has largely been applied in X-ray and magnetic resonance imaging for medical purposes [51], to automate the segmentation [52,53,54], for classification purposes [55,56,57], or, recently, applied in microscopic imaging where the potential for DL technologies is unprecedented and has still not reached its full potential [58]. Viruses, bacteria, parasites, and fungi can be monitored, investigated, and classified by the attribution of features (geometric characteristics) on microscopic images used as input in the DL models [59,60]. Details are reported in Table 3.
The techniques listed in Table 1, Table 2 and Table 3 can be applied in cultural heritage and neutron applications.

2.3. Deep Learning in Neutron Imaging and Future Applications in Cultural Heritage

In Neutron Imaging, Deep Learning models are successfully applied in different stages of data processing from the acquisition to the data analysis.
Figure 2 shows a general architecture of Neutron Imaging studies based on Deep Learning.
In this framework, a selection of the most relevant works is presented in Table 4 [23,25,26,38].
To optimize the prediction performance, Qiao et al. [23] applied the Gradient Magnitude Similarity Deviation (GMSD) to label the distorted images with a quality score. The labeling could be very useful in cultural heritage, where one or more categories can be classified for several purposes: authentication, conservation, and production processes identification. Micieli et al. [60], indeed, applied iterative optimization methods such as Neural Network Filtered Back-Projection (NN-FBP) to reduce the acquisition, the scan, the reconstruction time, and the amount of data storage in neutron tomography experiments. The NN-FBP model mitigates problems, i.e., artefacts, of the classical analytical reconstruction algorithm of Filtered Back-Projection. In detail, they quantitatively compared the NN-FBP, FBP, and Simultaneous Iterative Reconstruction Technique (SIRT) methods as a function of the number of projections, and applied to a set of simulated data, a phantom of Cu-CuCrZr pipe, and the real data of the pipe. The evaluation of the image quality was carried out by computing the Normalized Root Mean Square Error (NRMSE), the Structural Similarity Index (SSIM), the Feature Similarity Index (FSIM), and the Gradient Magnitude Similarity Deviation (GMSD). Tomographic scans of two similar samples were collected, generating a dataset of 1335 projections in the angular range [0°, 360°] with 30 s of exposure time per projection, approximately 11 h for a complete CT scan. The NN-FBP model reveals that the number of projections can be reduced to 223, i.e., 1/6 of the over-sampled dataset and 1/3 of the projections required by the sampling theorem (Nyquist–Shannon). The applicability in the cultural heritage context is immediate: the activation—and as a consequence, the de-activation—of the samples, in particular for metallic artefacts, is one of the main problems that need to be controlled because it is in close connection with the timing dictated by the owners, such as the museums, that require short time of permanence in the neutron facility and non-invasive and non-destructive investigations.
Lee et al. [26] successfully applied the Generative Adversarial Network (GAN) to avoid the trade-off for both high-phase sensitivity and high-resolution in interferometric applications. The GAN is a good option for image generation. It involves two neural models with conflicting objectives, one generator (G), and one discriminator (D), forcing each other to improve the output image file to again be converted back to the image file which represents the file output of the model. In detail, to obtain an image that has high-phase sensitivity as well as high-spatial resolution, a match of two images at different positions is acquired: the first one with low-contrast and high-resolution, the second one with high-contrast and low-resolution. The GAN is applied to generate the final image, a good combination of the two input images. The training dataset in ref [26] is composed of 890 image pairs; each picture has a 2000 × 2000 pixels size. The variables are connected through convolution layers (3 × 3 kernels), batch normalization, and rectified linear unit (ReLU) function. The final image is then optimized by minimizing the mean square error (MSE) between the first output image (merge of the two images) and the ground-truth image (discriminator output). The combined images from the GAN model demonstrate that the resolution and the sensitivity have been improved even if the shape is complex. In the cultural heritage context, the GAN approach could facilitate the great problem of the irregular shapes of investigated ancient objects such as pottery, archaeological objects, or ancient objects whose original morphology has changed due to degradation processes.
The last-mentioned case from Aoki et al. [38] regards the possibility to identify hidden profiles, the structure of the surfaces, and the interfaces of various materials. The morphology of the damage present in the same multi-material object is a common example. The degradation processes have a different reactivity as a function of the interfaces between the various materials. The authors [38] propose network architectures that differentiate between noisy and clean images. In detail, different blocks with different layers of convolution, batch normalization, and rectified linear units are employed for the ground truth and simulated neutron reflectometry profiles improve the efficiency of the learning process, which improves the denoising performance. In Neutron Tomography (NT), the data processing is composed of different steps: the acquisition phase at different angles, where the high number of projections of the object is acquired at different viewing angles; rotating the object around a vertical axis in the beam; the reconstruction process carried out through dedicated algorithms, such as the filtered back projection; and finally, the 3D visualization and data analysis such as segmentation. Appropriate regularization/prior-model parameters for each experiment and data acquisition are also required [61]. The acquisition time per digital image with high quality can span from a few seconds to minutes. Figure 2 shows the phases of a typical NI experiment and the possible applications of DL, from data acquisition to data analysis and visualization.
Energy-resolved Neutron Imaging has developed in recent years owing to the use of bright pulsed neutron beamline facilities at neutron spallation sources and is applied to study the composition, strain, and texture of the sample as a function of the spatial resolution [62,63,64]. Because the data are affected by noise, Machine Learning could be successfully applied to cultural heritage optimizing the acquisition time and improving the quantification results.
Table 5 shows the DL applicable in the cultural heritage domain selecting the aim and the challenge addressed with the consequent improvement [65,66].
The automatic segmentation based on U-Net can extract effective information and improve the level of material damage due to automatic learning image features. U-Net can be widely used in the cultural heritage image analysis with multiple purposes: for the identification of the cracks, i.e., to study the microstructural level of ancient concrete or mortar, to study the de-cohesions or de-adhesions of the pictorial painting on the supports, to study the stratigraphic sections segmentation, and classification of morphologies of the decay at different levels such as the degradation due to soluble salts on different supports. The U-Net network extracts deeper image features by a large number of feature channels which allows the network to propagate contextual information to higher-resolution layers [68]. Patch-CNN is particularly resolved for feature extraction. The latter, characterized by convolution algorithms, are transferred in the hierarchical fusion model to then have as output the classifications into labels [69]. Spiking Neural Networks are already tested in medical imaging and can be used in the cultural heritage context for real-time implementations for the automation of experiments. In this context, real-time identification and monitoring of the minimum number of needed radiographies collected at different angles would increase the safety of the ancient objects during the experiment, minimizing the total neutron dose and reducing activation.

3. Conclusions: Future Insights and Perspectives

Neutron Imaging is a powerful tool for the analysis of cultural heritage objects to obtain spatial information about the inner structure, internal morphology, and volumes that could be used to determine manufacturing processes, provenance, dating, and information about the state of conservation. In this context, the Deep Learning approach was applied in the data pre-processing phase to merge radiographs obtained by different set-ups and for the identification of the required number of radiographic images to obtain a successful reconstruction. Applications of Deep Learning in the imaging analysis phase, such as automatic segmentation, can give new ideas and perspectives. The common elements of the application of these algorithms are employed to reduce the dimensionality of the data, extract useful features to classify objects, and obtain the best images possible through optimization processes.
Cultural heritage objects are generally made of complex materials. As such, the activation of the samples and time irradiation need to be reduced as much as possible through optimization procedures to guarantee the return of ancient objects to the museum in a short time. All the experiments, indeed, need to be non-destructive and non-invasive. Moreover, these objects are characterized by a complex shape and often by a multi-material composition and get more difficult during the analysis phase. In particular, the available automatic segmentation procedures based on the grey scale could be enhanced by recognition algorithms such as semantic computational models that can classify the shape and pattern of the dataset by metric labelling. Therefore, Convolutional Neural Networks and new algorithms have been demonstrated as effective to classify the defect in metals, in concrete materials, recognize structural defects, and extract features on paintings, and can be a good candidate for future applications in Neutron Imaging. Moreover, there are new, exciting perspectives in the application of the Convolutional Neural Networks and Spiking Neural Networks architectures to solve most of the common problems in the Neutron Imaging applications in cultural heritage, from solving the problem of detection and quantification of the damaged products by automatic segmentation to estimating with high-resolution infiltration problems of ancient architectural monuments. In conclusion, Machine Learning and Deep Learning models can give new perspectives and contributions in the future for cultural heritage Neutron Imaging applications.

Author Contributions

Conceptualization, C.S. and G.F. (lead); writing—original draft preparation, C.S.; writing—review and editing, C.S. and G.F.; visualization, C.S. and G.F.; supervision, G.F.; project administration, G.F. All authors have read and agreed to the published version of the manuscript.

Funding

CREF—Museo Storico della Fisica e Centro Studi e Ricerche Enrico Fermi founding in the framework of the Physics for Cultural Heritage research (post-doctoral PhD, Prot. n. 0000178(22)).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

Claudia Scatigno thanks the CREF for her post-doctoral Ph.D. (Prot. n. 0000178(22)).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chitradevi, B.; Srimathi, P. An overview on image processing techniques. Int. J. Innov. Res. Comput. Commun. Eng. 2014, 2, 6466–6472. [Google Scholar]
  2. Danan, E.; Shabairou, N.; Danan, Y.; Zalevsky, Z. Signal-to-Noise Ratio Improvement for Multiple-Pinhole Imaging Using Supervised Encoder–Decoder Convolutional Neural Network Architecture. Photonics 2022, 9, 69. [Google Scholar] [CrossRef]
  3. Kardjilov, N.; Festa, G. (Eds.) Neutron Methods for Archaeology and Cultural Heritage; Springer: Berlin, Germany, 2017. [Google Scholar]
  4. Festa, G.; Romanelli, G.; Senesi, R.; Arcidiacono, L.; Scatigno, C.; Parker, S.F.; Marques, M.P.M.; Andreani, C. Neutrons for Cultural Heritage—Techniques, Sensors, and Detection. Sensors 2020, 20, 502. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Mannes, D.; Lehmann, E.; Masalles, A.; Schmidt-Ott, K.; Schaeppi, K.; Schmid, F.; Peetermans, S.; Hunger, K. The study of cultural heritage relevant objects by means of neutron imaging techniques. Insight—Non Destr. Test. Cond. Monit. 2014, 56, 137. [Google Scholar] [CrossRef]
  6. Kockelmann, W.; Siano, S.; Bartoli, L.; Visser, D.; Hallebeek, P.; Traum, R.; Linke, R.; Schreiner, M.; Kirfel, A. Applications of TOF neutron diffraction in archaeometry. Appl. Phys. A Mater. Sci. Process 2006, 83, 175–182. [Google Scholar] [CrossRef]
  7. Kak, A.C.; Slaney, M. Principles of computerized tomographic imaging. Society for Industrial and Applied Mathematics, Philadelphia; Stewart P. Cold neutron imaging for gas turbine inspection. Real Time Radiol. Imag. 2001, 8, 180–198. [Google Scholar]
  8. Peetermans, S.; Lehmann, E. Simultaneous neutron transmission and diffraction contrast tomography as a non-destructive 3D method for bulk single crystal quality investigations. J. Appl. Phys. 2013, 114, 124905. [Google Scholar] [CrossRef] [Green Version]
  9. Pfeiffer, F.; Grünzweig, C.; Bunk, O.; Frei, G.; Lehmann, E.; David, C. Neutron phase imaging and tomography. Phys. Rev. Lett. 2006, 96, 215505. [Google Scholar] [CrossRef] [Green Version]
  10. Festa, G.; Andreani, C.; De Pascale, M.P.; Senesi, R.; Vitali, G.; Porcinai, S.; Giusti, A.M.; Schulze, R.; Canella, L.; Kudejova, P.; et al. A nondestructive stratigraphic and radiographic neutron study of Lorenzo Ghiberti’s reliefs from paradise and north doors of Florence baptistery. J. Appl. Phys. 2009, 106, 074909. [Google Scholar] [CrossRef] [Green Version]
  11. Festa, G.; Christiansen, T.; Turina, V.; Borla, M.; Kelleher, J.; Arcidiacono, L.; Cartechini, L.; Ponterio, R.C.; Scatigno, C.; Senesi, R.; et al. Egyptian metallic inks on textiles from the 15th century BCE unravelled by non-invasive techniques and chemometric analysis. Sci. Rep. 2019, 9, 7310. [Google Scholar] [CrossRef] [Green Version]
  12. Andreani, C.; Aliotta, F.; Arcidiacono, L.; Borla, M.; Di Martino, D.; Facchetti, F.; Ferraris, E.; Festa, G.; Gorini, G.; Kockelmann, W.; et al. A neutron study of sealed pottery from the grave-goods of Kha and Merit. J. Anal. At. Spectrom. 2017, 32, 1342–1347. [Google Scholar] [CrossRef]
  13. Leona, M.; Fukunaga, K.; Liang, H.; Baglioni, P.; Festa, G.; Levchenko, V. From physics to art and back. Nat. Rev. Phys. 2021, 3, 681–684. [Google Scholar] [CrossRef]
  14. Kardjilov, N.; Manke, I.; Woracek, R.; Hilger, A.; Banhart, J. Advances in neutron imaging. Mater. Today 2018, 21, 652–672. [Google Scholar] [CrossRef]
  15. Schillinger, B.; Beaudet, A.; Fedrigo, A.; Grazzi, F.; Kullmer, O.; Laaß, M.; Makowska, M.; Werneburg, I.; Zanolli, C. Neutron imaging in cultural heritage research at the FRM II reactor of the Heinz Maier-Leibnitz center. J. Imaging 2018, 4, 22. [Google Scholar] [CrossRef] [Green Version]
  16. Laaß, M.; Hampe, O.; Schudack, M.; Hoff, C.; Kardjilov, N.; Hilger, A. New insights into the respiration and metabolic physiology of Lystrosaurus. Acta Zool. 2011, 92, 363–371. [Google Scholar] [CrossRef]
  17. Zanolli, C.; Schillinger, B.; Beaudet, A.; Kullmer, O.; Macchiarelli, R.; Mancini, L.; Schrenk, F.; Tuniz, C.; Vodopivec, V. Exploring hominin and non-hominin primate dental fossil remains with neutron microtomography. Phys. Procedia 2017, 88, 109–115. [Google Scholar] [CrossRef]
  18. Laaß, M.; Schillinger, B.; Kaestner, A. What did the “Unossified zone” of the non-mammalian therapsid braincase house? J. Morphol. 2017, 278, 1020–1032. [Google Scholar] [CrossRef]
  19. Trtik, P.; Hovind, J.; Grünzweig, C.; Bollhalder, A.; Thominet, V.; David, C.; Kaestner, A.; Lehmann, E.H. Improving the spatial resolution of neutron imaging at paul scherrer institut–the neutron microscope project. Phys. Procedia 2015, 69, 169–176. [Google Scholar] [CrossRef] [Green Version]
  20. Wu, H.; Khaykovich, B.; Wang, X.; Hussey, D.S. Wolter mirrors for neutron imaging. Phys. Procedia 2017, 88, 184–189. [Google Scholar] [CrossRef]
  21. Hussey, D.S.; Wen, H.; Wu, H.; Gentile, T.R.; Chen, W.; Jacobson, D.L.; LaManna, J.M.; Khaykovich, B. Demonstration of focusing Wolter mirrors for neutron phase and magnetic imaging. J. Imaging 2018, 4, 50. [Google Scholar] [CrossRef] [Green Version]
  22. Lehmann, E.H.; Hartmann, S.; Speidel, M.O. Investigation of the content of ancient Tibetan metallic Buddha statues by means of neutron imaging methods. Archaeometry 2010, 52, 416–428. [Google Scholar] [CrossRef]
  23. Qiao, S.; Li, J.; Zhao, C.; Zhang, T. No-reference quality assessment for neutron radiographic image based on a deep bilinear convolutional neural network. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip. 2021, 1005, 165406. [Google Scholar] [CrossRef]
  24. Festa, G.; Andreani, C.; Arcidiacono, L.; Burca, G.; Kockelmann, W.; Minniti, T.; Senesi, R. Characterization of γ-ray background at IMAT beamline of ISIS Spallation Neutron Source. J. Instrum. 2017, 12, 08005. [Google Scholar] [CrossRef] [Green Version]
  25. Miceli, A.; Festa, G.; Senesi, R.; Cippo, E.P.; Giacomelli, L.; Tardocchi, M.; Scherillo, A.; Schooneveld, E.; Frost, C.; Gorini, G.; et al. Measurements of gamma-ray background spectra at spallation neutron source beamlines. J. Anal. At. Spectrom. 2014, 29, 1897–1903. [Google Scholar] [CrossRef] [Green Version]
  26. Lee, S.; Oh, O.; Kim, Y.; Kim, D.; Hussey, D.S.; Wang, G.; Lee, S.W. Deep learning for high-resolution and high-sensitivity interferometric phase contrast imaging. Sci. Rep. 2020, 10, 9891. [Google Scholar] [CrossRef]
  27. Llamas, J.; Lerones, P.M.; Zalama, E.; Gómez-García-Bermejo, J. Applying Deep Learning Techniques to Cultural Heritage Images within the INCEPTION Project; Springer: Cham, Switzerland, 2016; pp. 25–32. [Google Scholar]
  28. Pierdicca, R.; Paolanti, M.; Matrone, F.; Martini, M.; Morbidoni, C.; Malinverni, E.S.; Frontoni, E.; Lingua, A.M. Point Cloud Semantic Segmentation Using a Deep Learning Framework for Cultural Heritage. Remote Sens. 2020, 12, 1005. [Google Scholar] [CrossRef] [Green Version]
  29. Garcia-Garcia, A.; Orts-Escolano, S.; Oprea, S.; Villena-Martinez, V.; Garcia-Rodriguez, J. A review on deep learning techniques applied to semantic segmentation. arXiv 2017, arXiv:1704.06857. [Google Scholar]
  30. Scatigno, C.; Festa, G. FTIR coupled with machine learning to unveil spectroscopic benchmarks in the Italian EVOO. Int. J. Food Sci. Technol. 2022, 57, 4156–4162. [Google Scholar] [CrossRef]
  31. Nunes, J.D.; Carvalho, M.; Carneiro, D.; Cardoso, J.S. Spiking Neural Networks: A Survey. IEEE Access 2022, 10, 60738–60764. [Google Scholar] [CrossRef]
  32. Cao, Y.; Chen, Y.; Khosla, D. Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. 2015, 113, 54–66. [Google Scholar] [CrossRef]
  33. Yuan, X.; Han, S. Single-pixel neutron imaging with artificial intelligence: Breaking the barrier in multi-parameter imaging, sensitivity, and spatial resolution. Innovation 2021, 2, 100100. [Google Scholar] [CrossRef] [PubMed]
  34. Zhang, X.; Saniie, J.; Heifetz, A. Detection of defects in additively manufactured stainless steel 316L with a compact infrared camera and machine learning algorithms. JOM 2020, 72, 4244–4253. [Google Scholar] [CrossRef]
  35. Butler, K.T.; Davies, D.W.; Cartwright, H.; Isayev, O.; Walsh, A. Machine learning for molecular and materials science. Nature 2018, 559, 547–555. [Google Scholar] [CrossRef]
  36. Tembely, M.; AlSumaiti, A.M.; Alameri, W.S. Machine and deep learning for estimating the permeability of complex carbonate rock from X-ray micro-computed tomography. Energy Rep. 2021, 7, 1460–1472. [Google Scholar] [CrossRef]
  37. Jia, Y.; Yu, G.; Du, J.; Gao, X.; Song, Y.; Wang, F. Adopting traditional image algorithms and deep learning to build the finite model of a 2.5 D composite based on X-Ray computed tomography. Compos. Struct. 2021, 275, 114440. [Google Scholar] [CrossRef]
  38. Aoki, H.; Liu, Y.; Yamashita, T. Deep learning approach for an interface structure analysis with a large statistical noise in neutron reflectometry. Sci. Rep. 2021, 11, 22711. [Google Scholar] [CrossRef]
  39. Turkson, R.E.; Qu, H.; Mawuli, C.B.; Eghan, M.J. Classification of Alzheimer’s disease using deep convolutional spiking neural network. Neural Process. Lett. 2021, 53, 2649–2663. [Google Scholar] [CrossRef]
  40. Ahmadi, M.; Sharifi, A.; Hassantabar, S.; Enayati, S. QAIS-DSNN: Tumor area segmentation of MRI image with optimized quantum matched-filter technique and deep spiking neural network. BioMed Res. Int. 2021, 2021, 6653879. [Google Scholar] [CrossRef]
  41. Garain, A.; Basu, A.; Giampaolo, F.; Velasquez, J.D.; Sarkar, R. Detection of COVID-19 from CT scan images: A spiking neural network-based approach. Neural Comput. Appl. 2021, 33, 12591–12604. [Google Scholar] [CrossRef]
  42. Dansana, D.; Kumar, R.; Bhattacharjee, A.; Hemanth, D.J.; Gupta, D.; Khanna, A.; Castillo, O. Early diagnosis of COVID-19-affected patients based on X-ray and computed tomography images using deep learning algorithm. Soft Comput. 2020, 26, 1–9. [Google Scholar] [CrossRef]
  43. Tajbakhsh, N.; Jeyaseelan, L.; Li, Q.; Chiang, J.N.; Wu, Z.; Ding, X. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Med. Image Anal. 2020, 63, 101693. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Amyar, A.; Modzelewski, R.; Li, H.; Ruan, S. Multi-task deep learning based CT imaging analysis for covid-19 pneumonia: Classification and segmentation. Comput. Biol. Med. 2020, 126, 104037. [Google Scholar] [CrossRef]
  45. Kobayashi, K.; Hwang, S.-W.; Okochi, T.; Lee, W.-H.; Sugiyama, J. Non-destructive method for wood identification using conventional X-ray computed tomography data. J. Cult. Herit. 2019, 38, 88–93. [Google Scholar] [CrossRef]
  46. Bouibed, M.L.; Nemmour, H.; Chibani, Y. SVM-based writer retrieval system in handwritten document images. Multimed. Tools Appl. 2021, 81, 22629–22651. [Google Scholar] [CrossRef]
  47. Massimiliano, P.; Alfio, V.S.; Costantino, D.; Scaringi, D. Data for 3D reconstruction and point cloud classification using machine learning in cultural heritage environment. Data Brief 2022, 42, 108250. [Google Scholar]
  48. Kambau, R.A.; Hasibuan, Z.A.; Pratama, M.O. Classification for multiformat object of cultural heritage using deep learning. In Proceedings of the 2018 Third International Conference on Informatics and Computing (ICIC), Palembang, Indonesia, 17–18 October 2018; pp. 1–7. [Google Scholar]
  49. Cintas, C.; Lucena, M.; Fuertes, J.M.; Delrieux, C.; Navarro, P.; González-José, R.; Molinos, M. Automatic feature extraction and classification of Iberian ceramics based on deep convolutional networks. J. Cult. Herit. 2020, 41, 106–112. [Google Scholar] [CrossRef]
  50. Lu, P.; Peng, X.; Yu, J.; Peng, X. Gated CNN for visual quality assessment based on color perception. Signal Process. Image Commun. 2019, 72, 105–112. [Google Scholar] [CrossRef]
  51. Rachmadi, M.F.; Valdés-Hernández, M.d.C.; Agan, M.L.F.; Komura, T. Deep learning vs. conventional machine learning: Pilot study of wmh segmentation in brain MRI with absence or mild vascular pathology. J. Imaging 2017, 3, 66. [Google Scholar] [CrossRef] [Green Version]
  52. Klöppel, S.; Abdulkadir, A.; Hadjidemetriou, S.; Issleib, S.; Frings, L.; Thanh, T.N.; Mader, I.; Teipel, S.J.; Hüll, M.; Ronneberger, O. A comparison of different automated methods for the detection of white matter lesions in MRI data. NeuroImage 2011, 57, 416–422. [Google Scholar] [CrossRef]
  53. Leite, M.; Rittner, L.; Appenzeller, S.; Ruocco, H.H.; Lotufo, R. Etiology-based classification of brain white matter hyperintensity on magnetic resonance imaging. J. Med. Imaging 2015, 2, 014002. [Google Scholar] [CrossRef] [Green Version]
  54. Ithapu, V.; Singh, V.; Lindner, C.; Austin, B.P.; Hinrichs, C.; Carlsson, C.M.; Bendlin, B.B.; Johnson, S.C. Extracting and summarizing white matter hyperintensities using supervised segmentation methods in Alzheimer’s disease risk and ageing studies. Hum. Brain Mapp. 2014, 35, 4219–4235. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  55. Kamnitsas, K.; Ledig, C.; Newcombe, V.F.; Simpson, J.P.; Kane, A.D.; Menon, D.K.; Rueckert, D.; Glocker, B. Efficient multi-scale 3D {CNN} with fully connected {CRF} for accurate brain lesion segmentation. Med. Image Anal. 2017, 36, 61–78. [Google Scholar] [CrossRef] [PubMed]
  56. Brosch, T.; Tang, L.Y.; Yoo, Y.; Li, D.K.; Traboulsee, A.; Tam, R. Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans. Med. Imaging 2016, 35, 1229–1239. [Google Scholar] [CrossRef] [PubMed]
  57. Brosch, T.; Yoo, Y.; Tang, L.Y.; Li, D.K.; Traboulsee, A.; Tam, R. Deep convolutional encoder networks for multiple sclerosis lesion segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015; pp. 3–11. [Google Scholar]
  58. Zhang, Y.; Jiang, H.; Ye, T.; Juhas, M. Deep learning for imaging and detection of microorganisms. Trends Microbiol. 2021, 29, 569–572. [Google Scholar] [CrossRef] [PubMed]
  59. Lugagne, B.; Lin, H.; Dunlop, M.J. DeLTA: Automated cell segmentation, tracking, and lineage reconstruction using deep learning. PLoS Comput. Biol. 2020, 16, e1007673. [Google Scholar] [CrossRef] [Green Version]
  60. Micieli, D.; Minniti, T.; Evans, L.M.; Gorini, G. Accelerating Neutron tomography experiments through Artificial Neural Network based reconstruction. Sci. Rep. 2019, 9, 2450. [Google Scholar] [CrossRef] [Green Version]
  61. Venkatakrishnan, S.; Ziabari, A.; Hinkle, J.; Needham, A.W.; Warren, J.M.; Bilheux, H.Z. Convolutional neural network based non-iterative reconstruction for accelerating neutron tomography. Mach. Learn. Sci. Technol. 2021, 2, 025031. [Google Scholar] [CrossRef]
  62. Lehmann, E.H.; Peetermans, S.; Josic, L.; Leber, H.; van Swygenhoven, H. Energy-selective neutron imaging with high spatial resolution and its impact on the study of crystalline-structured materials. Nucl. Instrum. Methods Phys. Res. Sect. A Accel. Spectrometers Detect. Assoc. Equip. 2014, 735, 102–109. [Google Scholar] [CrossRef]
  63. Kamiyama, T.; Hirano, K.; Sato, H.; Ono, K.; Suzuki, Y.; Ito, D.; Saito, Y. Application of Machine Learning Methods to Neutron Transmission Spectroscopic Imaging for Solid–Liquid Phase Fraction Analysis. Appl. Sci. 2021, 11, 5988. [Google Scholar] [CrossRef]
  64. Venkatakrishnan, S.; Zhang, Y.; Dessieux, L.; Hoffmann, C.; Bingham, P.; Bilheux, H. Improved Acquisition and Reconstruction for Wavelength-Resolved Neutron Tomography. J. Imaging 2021, 7, 10. [Google Scholar] [CrossRef]
  65. Mannes, D.; Lehmann, E.H. Neutron Imaging of Cultural Heritage Objects. In Handbook of Cultural Heritage Analysis; Springer International Publishing: Cham, Switzerland, 2022; pp. 211–237. [Google Scholar]
  66. Xu, K.; Tremsin, A.S.; Li, J.; Ushizima, D.M.; Davy, C.A.; Bouterf, A.; Su, Y.T.; Marroccoli, M.; Mauro, A.M.; Osanna, M.; et al. Microstructure and water absorption of ancient concrete from Pompeii: An integrated synchrotron microtomography and neutron radiography characterization. Cem. Concr. Res. 2021, 139, 106282. [Google Scholar] [CrossRef]
  67. Rymarczyk, T.; Kłosowski, G.; Hoła, A.; Sikora, J.; Wołowiec, T.; Tchórzewski, P.; Skowron, S. Comparison of machine learning methods in electrical tomography for detecting moisture in building walls. Energies 2021, 14, 2777. [Google Scholar] [CrossRef]
  68. Du, G.; Cao, X.; Liang, J.; Chen, X.; Zhan, Y. Medical image segmentation based on u-net: A review. J. Imaging Sci. Technol. 2020, 64, 1–12. [Google Scholar] [CrossRef]
  69. Zhang, M.; Li, W.; Du, Q.; Gao, L.; Zhang, B. Feature extraction for classification of hyperspectral and LiDAR data using patch-to-patch CNN. IEEE Trans. Cybern. 2018, 50, 100–111. [Google Scholar] [CrossRef] [PubMed]
Jimaging 08 00284 g001 550
Figure 1. Overview of the main in Neutron Imaging applied to cultural heritage.
Figure 1. Overview of the main in Neutron Imaging applied to cultural heritage.
Jimaging 08 00284 g001
Jimaging 08 00284 g002 550
Figure 2. General overview of the Deep Learning architecture in Neutron Imaging applications. Training datasets that correspond to labelled variables are able to validate new datasets where the variables are still unlabelled, predicting the clustering of belonging by the similarity of their distribution along the regression line.
Figure 2. General overview of the Deep Learning architecture in Neutron Imaging applications. Training datasets that correspond to labelled variables are able to validate new datasets where the variables are still unlabelled, predicting the clustering of belonging by the similarity of their distribution along the regression line.
Jimaging 08 00284 g002
Table
Table 1. Three examples of Machine Learning and Deep Learning techniques applied in X-ray tomography imaging.
Table 1. Three examples of Machine Learning and Deep Learning techniques applied in X-ray tomography imaging.
MethodOperationAimImprovementLimitation
CNNFeature extraction,
modeling, and fine-tuning		Early detect diseaseNoise reductionPreprocessing phase
SVR *, LR *, RF *, IGB *, CNN Feature extractionPredict petrophysical properties Computation time
reduction		A large calibration dataset is required
GLCM *, LBPs *, k-NN *Feature extraction and
classification		Material identificationClassification accuraciesPretreatment data
* SVR: Support Vector Regression, LR: Linear Regression, RF: Random Forest, IGB: Improved Gradient Boosting, GLCM: grey-level co-occurrence matrix, LBPs: Local Binary Patterns, k-NN: K-Nearest Neighbor.
Table
Table 2. ML and DL techniques applied in CH imaging.
Table 2. ML and DL techniques applied in CH imaging.
MethodDatasetAimLimitationCH in NI
SVMDigitized documentsHandwriting recognitionNo replicability for other datasetsJimaging 08 00284 i001
RFPhotogrammetric
images		Features classification and segmentation for 3D reconstructionSeveral stepsJimaging 08 00284 i001
CNNBinary profile imagesPottery classificationResizing imagesJimaging 08 00284 i001
Table
Table 3. Algorithms used in microbiology, with their usability and the corresponding improvement in the field.
Table 3. Algorithms used in microbiology, with their usability and the corresponding improvement in the field.
MethodAimImprovementChallenge
CNNsAutomatic detection of
microorganisms		Accuracy (quantification) and speed of diagnosis Transforming lower resolution into super-resolution images and constructing 3D images such as with fluorescence microscopy
Virus classification
(structure, size, and morphology)		Fast and cost-efficient classification
U-NetSegmentation of imagesDetection and counting of colonies
Table
Table 4. Deep Learning algorithms used in different Neutron Imaging techniques are listed, with their usability and the corresponding Neutron Imaging in cultural heritage applicability with the improvement in the field.
Table 4. Deep Learning algorithms used in different Neutron Imaging techniques are listed, with their usability and the corresponding Neutron Imaging in cultural heritage applicability with the improvement in the field.
MethodNI TechniqueAimApplicability to CH-NI Improvement
CNN-GMSD *RadiographyExtracting featuresJimaging 08 00284 i002Classification of materials
NN-FBP *TomographyReducing data acquisition time and data
storage		Jimaging 08 00284 i002Especially for active materials
CNN-GAN *Interferometric
phase-contrast imaging		Improving resolution and the sensitivityJimaging 08 00284 i002Shapes complex
pCNN, DnCNN, CAE *ReflectometryExtracting hidden
profiles in large
statistical noise		Jimaging 08 00284 i002Multimaterial, different shapes
* GMSD: Gradient Magnitude Similarity Deviation, NN-FBP: Neural Network Filtered Back-Projection, GAN: Generative Adversarial Network, pCNN: Plain CNN, DnCNN: Denoising CNN. CAE: Convolutional Autoencoder.
Table
Table 5. NT and NR in cultural heritage and the improvements obtained via DL algorithms.
Table 5. NT and NR in cultural heritage and the improvements obtained via DL algorithms.
Material InvestigatedAimChallengeDL ApplicableImprovement
Metallic statue Origin, purpose, manufacturing process, provenanceQuantitative purpose (filled voxels are hardly considered)CNNs/U-NetAutomatic counting of the voxels
restoration and conservation purpose (distinguishing different corrosion zones)Classification of
different corrosion products		CNNs/Patch-CNNDetection and quantification of the damaged products by automatic segmentation
Ancient concreteStudying the microstructure, cracks, and water adsorptionDynamics of water penetrationU-Net/SBL *High-resolution estimation
* SBL: Sparse Bayesian Learning [67].
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Scatigno, C.; Festa, G. Neutron Imaging and Learning Algorithms: New Perspectives in Cultural Heritage Applications. J. Imaging 2022, 8, 284. https://doi.org/10.3390/jimaging8100284

AMA Style

Scatigno C, Festa G. Neutron Imaging and Learning Algorithms: New Perspectives in Cultural Heritage Applications. Journal of Imaging. 2022; 8(10):284. https://doi.org/10.3390/jimaging8100284

Chicago/Turabian Style

Scatigno, Claudia, and Giulia Festa. 2022. "Neutron Imaging and Learning Algorithms: New Perspectives in Cultural Heritage Applications" Journal of Imaging 8, no. 10: 284. https://doi.org/10.3390/jimaging8100284

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop