An Intelligent DOA Estimation Error Calibration Method Based on Transfer Learning
Round 1
Reviewer 1 Report
s_K(t) in Eq. (5) should be s_P(t), because the explanation of S(t) in l. 115 (just below the equation (6).) It said that the S(t) is PX1 dimensional vector.
The unit of the vertical axis in Fig. 4 is not clear. It could be the angle in degree or radian. Please add the unit in the figure.
The results in Section 5 showed that the proposed method is capable of estimating the angle. However, the comparison among the conventional algorithms such as MUSIC with calibrations was not shown in the manuscript. Please discuss the performance against the conventional algorithms.
Minor comments: The functions such as cos in Eq. (2) and exp in Eq. (8) should be in roman font.
Author Response
Point 1: s_K(t) in Eq. (5) should be s_P(t), because the explanation of S(t) in l. 115 (just below the equation (6).) It said that the S(t) is PX1 dimensional vector.
Response 1: Thanks for the comment. We have modified s_K(t) to s_P(t) in Eq. (5).
Point 2: The unit of the vertical axis in Fig. 4 is not clear. It could be the angle in degree or radian. Please add the unit in the figure.
Response 2: Thanks for the comment. We have added the unit “rad” (radian) of the vertical axis in Fig. 4.
Point 3: The results in Section 5 showed that the proposed method is capable of estimating the angle. However, the comparison among the conventional algorithms such as MUSIC with calibrations was not shown in the manuscript. Please discuss the performance against the conventional algorithms.
Response 3: Thanks for the comment. It is difficult for us to compare the performance with traditional DOA estimation algorithms. Traditional DOA estimation algorithms have many restrictions in actual application. For example, The MUSIC algorithm is used under the following conditions: the signal sources are irrelevant, the number of signal sources is smaller than the array elements, the noise is additive Gaussian white noise, etc. However, the actual signal samples we collected from the actual environment have many error factors: the inconsistent amplitude and phase of array element channels, the mutual coupling between array elements, the uncertainty in the position of array elements, etc. Our method can learn all the error knowledge during transfer learning.
Point 4: Minor comments: The functions such as cos in Eq. (2) and exp in Eq. (8) should be in roman font.
Response 4: Thanks for the comment. We have modified the font of “cos” and “exp” to roman in Eq. (2) and Eq. (8).
Author Response File: 
Author Response.pdf
Reviewer 2 Report
1. In the introduction conventional method of DOA estimation were studied by using CNN. Why use CNN in the proposed method?
2. Normally when using transfer method in DL, models are trained in large dataset. However, in the study small number of samples were used to train the model. What methods were used to prevent overfitting?
3. Learning parameter between before transfer learning and after transfer learning should be shown and the computational advantage of after transfer learning compared to before transfer learning should be presented.
Author Response
Response to Reviewer 2 Comments
Point 1: In the introduction conventional method of DOA estimation were studied by using CNN. Why use CNN in the proposed method?
Response 1: Thanks for the comment. We use the covariance matrix of the phase difference between elements as the input. The covariance matrix can be regarded as a direction image for the input of the CNN. It not only contains all information about the direction of incident signal, but also eliminates some noise and interference. In our early work “Broadband Direction of Arrival Estimation Based on Convolutional Neural Network”, we explained in detail why CNN is used to process array signals. We have added some content about why using CNN (line 157 to line 160 in the revised manuscript).
Point 2: Normally when using transfer method in DL, models are trained in large dataset. However, in the study small number of samples were used to train the model. What methods were used to prevent overfitting?
Response 2: Thanks for the comment. We agree with your comment that “Normally when using transfer method in DL, models are trained in large dataset”. Because it is very difficult to collect sufficient actual signal samples for training, many of the existing DNN-based methods train models on simulation signal samples that are generated by the computer. We generate a large number of (72000) signal samples by computer simulation. The 72000 simulation signal samples are used to train the CNN-based intelligent DOA estimation model. Then 1800 actual signal samples we collected are used to fine tune the CNN-based model based on transfer learning method. During transfer learning, the learning rate is one tenth of that before. The CNN-based model learns complex actual error knowledge from 1800 actual signal samples. In conclusion, the CNN-based DOA estimation model we proposed is first trained on a simulated large dataset, and then fine tuned on the actual small dataset with a smaller learning rate. We prevent overfitting in this way.
Point 3: Learning parameter between before transfer learning and after transfer learning should be shown and the computational advantage of after transfer learning compared to before transfer learning should be presented.
Response 3: Thanks for the comment. We have added a description of the learning parameter before and after transfer learning (line 262 to line 265 in the revised manuscript).
“Before transfer learning, the batch size is set to 360, the learning rate is set to 0.001. We use the early stop strategy for training. When the validation error does not decrease for 5 epochs, the training will be stopped. When transfer learning, we fine tune the model with a smaller learning rate of 0.0001, other parameters remain unchanged.”
Author Response File: Author Response.pdf
Reviewer 3 Report
The authors need to do thorough proofreading of this paper. Some phrases don't make sense and there are lots of typos and wrong use of prepositions.
In the introduction, the findings of the present research work should be compared with recent work in the same field towards claiming the contribution made.
Authors can focus more on the discussion related to Figure 4. Phase difference of ideal simulation signal and actual signal between the first and second elements.
It is good to see the outcome of this work, but, the authors need to specify how it compares to other existing schemes.
Literature review techniques have to be strengthened by including the issues in the current system and how the author proposes to overcome the same.
Authors should discuss in detail on how Intelligent DOA Estimation Error Calibration Method Based on Transfer Learning is successful over the other methods.
The conclusion should state the scope for future work.
Discuss future intentions in light of the current state of research and its constraints.
Authors may refer to some more recent related works
Optimized Feature Extraction for Precise Sign Gesture Recognition Using Self-improved Genetic Algorithm
Analysis of dimensionality reduction techniques on big data
Author Response
Response to Reviewer 3 Comments
Point 1: The authors need to do thorough proofreading of this paper. Some phrases don't make sense and there are lots of typos and wrong use of prepositions.
Response 1: Thanks for the comment. We have proofread our manuscript and corrected some of the grammatical errors.
Point 2: In the introduction, the findings of the present research work should be compared with recent work in the same field towards claiming the contribution made.
Response 2: Thanks for the comment. We have added a detailed description of the issue in recent work and our contributions (line 37 to line 64 in the revised manuscript).
- issue: “However, the existing deep learning based methods mostly work in ideal conditions. Due to the various error factors in the actual environment, the accuracy of DOA estimation in practical applications of these methods will be greatly reduced. In order to make DNN-based DOA estimation models work well in the actual environment, a large number of actual signal samples need to be collected, however this is very difficult. In addition, when the environment changes, a large number of samples need to be re-collected and the model needs to be re-trained. Therefore, it is a great challenge to quickly calibrate errors and to make the DNN-based models efficiently applicable in the actual environment.”
- Our contribution: “To address the above issue, in this paper, we propose an intelligent DOA estimation error calibration method based on transfer learning. Transfer learning can update the trained model adaptively according to the new samples, and transfer the model to new tasks.The idea of transfer learning suits the task of DOA estimation. The large number of ideal signal samples generated by computer simulation can be used to train a DNN-based DOA estimation model to learn the mapping between signals and their azimuths. The small number of actual signal samples collected from the actual environment can be used to fine tune the model based on transfer learning. The original DOA estimation capability of the model is retained while the model can learn new error knowledge during transfer learning.
- In this way, only a small number of actual signal samples need to be collected, the model can calibrate the errors and be quickly applied in the actual environment. When the environment changes, the model can continuously learn the error knowledge in new environments by learning from a small number of new actual signal samples. ”
Point 3: Authors can focus more on the discussion related to Figure 4. Phase difference of ideal simulation signal and actual signal between the first and second elements.
Response 3: Thanks for the comment. We have added a detailed description of the error factors of ideal signal samples and actual signal samples. We also add the necessity of using transfer learning to achieve error calibration. (line 137 to line 149 in the revised manuscript)
“Because the signal in the actual environment is affected by many complex error factors, such as mutual coupling error, amplitude and phase error of array channel, array element position error, etc. These errors make the DOA estimation performance of the model decline sharply in the actual environment. Deep neural network (DNN) can learn the complex mapping relationship between signals and their azimuths. However, It is difficult to collect a large number of actual signal samples for training. Therefore, we propose the error calibration method based on transfer learning to make the DNN-based DOA estimation models can work well in the actual environment. The large number of ideal signal samples that generated by computer simulation are used to train a DNN-based model and learn the mapping between signals and their azimuths. The small number of actual signal samples are used to fine tune the model based on transfer learning and learn the complex error factors. In this way, the model is transferred from the ideal condition to the actual environment, and hence error calibration is achieved.”
Point 4: It is good to see the outcome of this work, but, the authors need to specify how it compares to other existing schemes.
Response 4: Thanks for the comment. The existing deep learning methods use DNNs to construct DOA estimation models. These methods are workable under ideal conditions and when used in a real-world environment, the accuracy of these methods is suffering due to complex error factors. The proposed method is the first deep learning based error calibration method, which learns error knowledge from a small number of actual signal samples based on transfer learning. Our method allows DNN-based DOA estimation models to be quickly calibrated for errors and applied to practical environments.
Point 5: Literature review techniques have to be strengthened by including the issues in the current system and how the author proposes to overcome the same.
Response 5: Thanks for the comment. The main issue is that the existing methods mostly work under ideal conditions and are difficult to apply in the actual environments. To address the issue, we adopt transfer learning method to learn the error knowledge from a small number of actual signal samples . We have strengthened our description about the issues and how we overcome. (line 44 to line 64 in the revised manuscript)
Point 6: Authors should discuss in detail on how Intelligent DOA Estimation Error Calibration Method Based on Transfer Learning is successful over the other methods.
Response 6: Thanks for the comment. Transfer learning can update the trained model adaptively according to the new samples, and transfer the model to new tasks. Therefore, the original DOA estimation capability of the model is retained while the model can learn new error knowledge during transfer learning. In this way, only a small number of actual signal samples need to be collected, the model can calibrate the errors and be quickly applied in the actual environment. When the environment changes, the model can continuously learn the error knowledge in new environments by learning from a small number of new actual signal samples. The existing DNN-based methods need a large number of actual samples to train the model and are not environmentally adaptable. (line 52 to line 64 in the revised manuscript)
Point 7: The conclusion should state the scope for future work.
Response 7: Thanks for the comment. We have added the future work in conclusion. (line 358 to line 361 in the revised manuscript)
Point 8: Discuss future intentions in light of the current state of research and its constraints.
Response 8: Thanks for the comment. Despite the preliminary results in this paper, however, fine tuning is a relatively simple approach of transfer learning. In future work, we will investigate more reasonable transfer learning approaches to make the model better learn the error knowledge in the actual environment.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
Points mentioned before were overall well explained. However in the results in table 5, "We can observe from Table 5 that with the increased number of actual samples used for transfer learning, the model performs better". Evaluation score presented when using 1440 samples shows the worst outcome. Moreover, the evaluation score did not improve as mentioned. Therefore, it is hard to believe that model performs better when using larger samples. More detailed explanation should be presented.
Author Response
Response : Thanks for the comment. Because a small number of actual signal samples are used to fine tune the model during transfer learning, the selection of a small number of samples has a large impact on the model performance. We conduct the experiments with only one set of randomly selected samples of different numbers and did not conduct multiple sets of experiments, so the experimental results may be affected by random errors. We appreciate your valuable comments. In order to eliminate random errors as much as possible, we repeat the experiment 5 times with different numbers of randomly selected samples, and the average of the 5 sets of experimental results is taken as the final experimental result. We can see from the new experimental results that MAE, RMSE, and MAXE keep decreasing overall and Ratio-1 keeps increasing as the actual sample number increases. The maximum angular error MAXE refers to the error of the one with the largest angular error among all the test samples, which is influenced more by some individual samples. Therefore it is reasonable for MAXE to have some volatility. In conclusion, from the new experimental results, we can conclude that "with the increased number of actual samples used for transfer learning, the model performs better". We have updated and corrected the experimental results in Table 5, which is also described in the revised manuscript. (line 338 to line 344 and line 346 to line 350)
Author Response File: Author Response.pdf
