Investigation into the Affect of Chemometrics and Spectral Data Preprocessing Approaches upon Laser-Induced Breakdown Spectroscopy Quantification Accuracy Based on MarSCoDe Laboratory Model and MarSDEEP Equipment

Round 1

Reviewer 1 Report

This paper compared the accuracy of PLS and BPNN models affected by the pre-processing steps, and provides an effective method to improve the quantification accuracy of LIBS spectra. It’s an interesting work but the manuscript cannot be accepted in its current form. However, it can be considered for publication if all the concerns are addressed. In addition, the English could be improved upon in the manuscript.

Main comments:

1.       Why MgO was selected to verify the quantification accuracy of LIBS spectra instead of FeO_T, SiO₂, et al.? These components are ubiquitous in Martian soils and rocks and are critical for identifying the types of Martian rocks.

2.       Please provide the RMSE of the training set. The comparison between RMSE values of the training and validation sets can provide constraints for the overfitting or underfitting of PLS and BPNN models.

3.       How do you complete the wavelength drift of Mg-lines? Which lines do you select for calibration? What’s the wavelength drift of each Mg-line? Please provide more details.

 

Minor comments:

1.       The unit of MgO abundance is wt.% or %? The unit “%” was used in line 206, but “wt.%” was in line 205, Table 2, and Table 3.

2.       Line 202, why do you remove samples with higher and lower MgO in the 2^nd set of samples? In addition, please provide the threshold value (i.e., what MgO abundance are these samples higher or lower than?)

3.       Line 266, for wavelength conversion and drift correction, what method do you choose to seek and match the Ti emission lines and the NIST database?

4.       Line 275, why the pixels at the end of CCD should be removed?

5.       Line 288, the background spectrum is caused by Bremsstrahlung and ion-electron recombination processes instead of Bremsstrahlung only.

6.       The defined background may be higher than expected near the LIBS emission lines. Thus, an additional concave feature can be observed in the spectrum after removing the continuous background (e.g., a concave feature is obvious near 760 nm in Fig. 3b). Please check and correct it.

7.       Line 408, which points are outliners in Fig. 6?

8.       Line 429, what does “five spectra” mean? Which five? Why do you only choose five spectra?

9.       Line 480, “Duo to” should be “Due to”.

10.     Line 491, “All kind of spectra” should be “All kinds of spectra”.

11.     Line 506, “Optimal” should be lowercase.

12.     When you compare the RMSE/RE of PLS and BPNN, please provide the RMES/RE values in the main text.

13.     There are so many syntax errors. Some sentences are composed of two or even more individual sentences but are joined by a comma. Please check the whole manuscript and correct them.

a. “Wavelength calibration is essential for quantitative analysis, four standard calibration lamps were used during wavelength calibration”.

b. “Calculate the standard deviation between the matched Mg peaks and the peaks in NIST database by translating the spectrum in each channel as a whole, the drift value with the minimum standard deviation was taken as the final result.”

c. “Generally, we use RMSE to evaluate the prediction ability of BPNN and PLS, which is the square root of a ratio, the ratio is the square sum of the deviations between all of the observation values and the true value divided by the number of observations M.”

d. “it can be seen that sample 2, 7 and 33 with low concentration got the largest RE values when using PLS to analyze, the MgO concentration of these samples are lower than 0.1%”

e. “There are 5 samples with prediction error less than 10%, it's the largest number for BPNN in range 1.”

f. “All kind of spectra have RE values in range 6, it causes a negative effect on the accuracy of quantification.”

g. “BPNN and PLS obtain the most accurate prediction results under this kind of spectrum, minimum and average values of RE are the Optimal results.”

h. “Too little information can be learned when using characteristic peaks for analysis, BPNN cannot take into account the impact of other elements on Mg.”

i. “Therefore, it is not suitable to use Mg-peak feature when using nonlinear method for analysis, it needs a lot of information and it is more suitable to use the whole spectrum to analyze.”

j. “For PLS, the analysis using the Mg-peak feature is similar to the original spectra, the use of the original spectra also depends on the Mg-peak strength.”

k. “but the impact of the two approaches on BPNN cannot be determined, further experimental verification is needed.”

l. “It can be seen that BPNN is not suitable for quantitative analysis by Mg-peak feature, there is too little information can be learned by BPNN in this circumstance.”

m. “It is not conducive to giving full play to the advantages of machine learning, BPNN cannot establish a good prediction model and it causes huge errors.”

n. “For the samples in this experiment, using the Mg-peak wavelength corrected spectra can improve the accuracy for PLS and BPNN, reducing concentration range can improve the prediction accuracy of PLS.”

Need some more improvment.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

In this work, the authors utilize various strategies for sample analysis that are applicable to the rovers currently exploring Mars. The simulation is carried out by placing the entire setup in a vacuum chamber with a simulated Martian atmosphere and a temperature of -16°C, which is suitable for the environment being simulated. While it is true that there are literally hundreds of works where different types of soils are analyzed with LIBS, this simulation applied under Martian conditions makes it useful for interpreting the data obtained by the probes on Mars. The work is interesting and could be considered for publication, but there are several points that need to be corrected.

The authors decided to use pressed powders instead of sintered powders, which is not clear whether it is a good simulation; in fact, in Ref [20], employs sintered powders. The authors must discuss this point in this paper since the LIBS signal strongly depends on the type of sample being analyzed. Detecting Mg in an alloy is not the same as detecting it in a rock or a compacted soil. Clearly, very different signals will be obtained, which greatly affects the quantification attempt.

 

Lines 89-100. The authors state that "The SuperCam team used ordinary least squares (OLS), random forest (RF), PLS, ridge regression (Ridge), elastic net (ENet) and other methods"… How do these methods compare with BPNN? This point should have been taken into account in this work. The implementation of these methods, since the data is preprocessed, is a matter of hours and would significantly improve the presented work.

Table 1: it is stated that the stand-off distance is 1.7-1.7 m, and the working distance in Figure 1 is indicated as 2 m. What is the actual distance from the mirror to the sample?

Section 2.3.4. It is mentioned that the intensity of each pixel is divided by the sum of the intensities of the entire spectrum. That means the intensity should be much lower than 1 for each pixel. However, Figure 3 shows intensities on the order of hundreds of counts. Can the authors clarify if the entire spectrum is then multiplied by a constant?

Figure 3: identify the most important lines. In addition to Mg, Na, O, Ca or Al, among other elements, can be observed. Was it possible to observe the carbon from CO2?

Section 2.4. The authors should develop in greater depth on the implemented methods. For example, what software was used for both PLS and BPNN? Was data normalization performed during PLS analysis?

In BPNN, why is a single hidden layer with 1024 neurons used? Was an optimization process done for this: why were 100 or 500 neurons or multiple layers not used? The more information provided, the better for reproducing the analysis in another laboratory.

Another important parameter is the number of iterations used: it is not stated whether it was 1000 or 100,000 or another number.

Were oscillations observed during the optimization process? I suggest changing the current Figure 4 to a plot of loss vs. rounds.

 Can the authors clarify in the text how many wavelengths were used? That is, whether both Mg lines were introduced into the calculation or only individually or the entire spectrum between 278-286 nm, for example. This needs to be clearly explained for the results shown in Table 4.

In section 2.3.7, it is mentioned that Mg peaks are extracted. Does this mean that the intensity of the lines is used? In that case, do the authors refer to the area or only the peak value of the lines? This must be clearly explained in the text.

Figure 6: Use a logarithmic scale on the y-axis to more clearly visualize the difference in error between both analysis methods.

The work in general is clear and I only found some details, such as the fact that the word "we" should be capitalized after the period in line 219. Additionally, it is preferable to use the passive voice instead of using "we" repeatedly (more than 60 times throughout the document).

Author Response

Please see the attachment.

Author Response File: Author Response.docx