Comparative Analysis of Discrete Subtraction and Cross-Correlation for Subpixel Object Tracking
Round 1
Reviewer 1 Report
The Mean Shift of the 3rd column and the 4th column in Table 3 are the same one, 0.0013. Why? In Table 3, STD is different. Question: From what samples to obtain these results?
Author Response
Reviewer #1
"The Mean Shift of the 3rd column and the 4th column in Table 3 are the same one, 0.0013. Why? In Table 3, STD is different. Question: From what samples to obtain these results?"
We assume you are referring to Table 1 (rather than Table 3), as the manuscript only contains two tables. In Table 1, we computed the interframe displacement, which represents the distance between an object's positions in two consecutive frames. In this case, the mean values for both methods (presented in the 3rd and 4th columns of Table 1) are identical because the linear motorized stage generates consistent movement, resulting in equal interframe displacements. These calculations were performed on recordings of the four objects depicted in Figure 1 (and named as “Square”, “Circle”, “Gaussian” and “Mule” in Table 1), with the linear motorized stage producing the same movement for all recordings.
It should be noted that while the mean value remains consistent across all cases, the standard deviation (STD), which accounts for noise and calculation inaccuracies, varies significantly. Dispersion may be taken as the true measure of accuracy, as values within the range defined by the mean and ±STD should be considered indistinguishable. Consequently, a high STD value indicates that the measurement of interframe displacement lacks precision, potentially leading to small drifts when the total movement is accumulated.
According to this, the standard deviation (STD) in the DSM method is larger compared to the cross-correlation method, which means that is more sensitive to noise. Also, this discrepancy was expected since, by definition, correlation minimizes the Mean Square Error, leading to lower dispersion than that observed in the DSM method. Furthermore, the standard deviation of the object's movement varies for each object due to their distinct shapes, which influence the ease or difficulty of detecting their positions in each case.
Since this may be difficult to interpret, a small explanation has been added to the text in lines 203 and following, just after Table 1 is presented.
Reviewer 2 Report
This paper presents further exploration on the discrete subtraction method. However, the paper lacks innovation. It might be experimental results about the current method. It is suggested to give details about the innovation. Additionally, there are some specific issues that need to be addressed:
1. The introduction should not include formulas. The detailed implementation of the methodology should be placed in the related work or other sections.
2. There are two periods in line 92.
3. In line 94, "paper" should be replaced with "manuscript." It is advisable to ensure consistency throughout the entire document.
4. The format of "1.5m" in line 99 is incorrect.
5. In line 211, I believe that an academic paper should not include such colloquial expressions, such as: ‘Let us also focus in Figure 7,’.
Minor editing of English language required
Author Response
Reviewer #2
This paper presents further exploration on the discrete subtraction method. However, the paper lacks innovation. It might be experimental results about the current method. It is suggested to give details about the innovation. Additionally, there are some specific issues that need to be addressed:
We disagree the reviewer in the fact that the paper lacks innovation because it describes experimental results but no new methods. The new methods that arise must be analyzed and compared to those already existing, and that is also innovation because it shows something unknown. Many times, the new method’s authors include some checks in their paper, but they are quite elemental and rough, due to the novelty of the method. Therefore, it is very common that new checks and studies done by other researcher groups arise around the new method (as we do in this paper), especially if the new method becomes popular. That is the case, for example of DIC to measure displacements; it is a well-known procedure from the 80’s but even nowadays it is not rare to find new studies trying to improve the method and comparing it to other procedures. If the DIC research would be finished in the 80’s, some important knowledge, as the peak locking for example, would not be known now.
However, we appreciate the reviewer for their suggestion. It appears that we may not have adequately explained the contribution and novelty of our work. In response to this feedback, we have thoroughly revised and rewritten the Abstract and Introduction sections to emphasize our specific contributions and the novelty of the research presented in this paper.
- The introduction should not include formulas. The detailed implementation of the methodology should be placed in the related work or other sections.
We acknowledge the reviewer's suggestion and have taken it into consideration. As a result, we have made the necessary adjustments to our paper. Specifically, we have included a comprehensive description of the methodology implementation in the Material and Methods section exclusively. Furthermore, we have subdivided this section into different subsections covering Theoretical Background, Experimental Setup, and Image Processing. This restructuring aims to enhance the readability and organization of the paper, allowing for a clearer understanding of the methodology.
- There are two periods in line 92.
It has been corrected.
- In line 94, "paper" should be replaced with "manuscript." It is advisable to ensure consistency throughout the entire document.
It has been replaced.
- The format of "1.5m" in line 99 is incorrect.
It has been reviewed.
- In line 211, I believe that an academic paper should not include such colloquial expressions, such as: ‘Let us also focus in Figure 7,’.
This expression has been rewritten in a less colloquial way. We thank the reviewer for its helpful suggestions regarding the writing style (from question 2 to 5), which have all been implemented and have improved the text understanding and style. Furthermore, we have thoroughly revised the entire text and made sure to rectify any other typos or colloquial expressions to the best of our knowledge.
Reviewer 3 Report
This paper proposes experimental comparison of two methods: discrete subtraction and cross-correlation for subpixel object tracking. In summary, the paper has certain application value, but its academic contributions are relatively small.
The proposed method applies the two mentioned methods in a real scenario where four different targets with apparent displacements as small as 0.002 px are considered. It is very likely that it has not been practiced before.
However, the methods are only directly applied without any innovation.
Some suggestions are as follows:
1. Diagrams need to be added to help readers better understand the implementation of the methods.
2. Principle analysis needs to be conducted to illustrate the results of the experiment, preferably mathematically.
Author Response
Reviewer #3
This paper proposes experimental comparison of two methods: discrete subtraction and cross-correlation for subpixel object tracking. In summary, the paper has certain application value, but its academic contributions are relatively small. The proposed method applies the two mentioned methods in a real scenario where four different targets with apparent displacements as small as 0.002 px are considered. It is very likely that it has not been practiced before. However, the methods are only directly applied without any innovation.
We respectfully disagree from the reviewer regarding their assessment of the academic contribution of the manuscript. Although this paper does not introduce any novel theory or method, it serves an important purpose by rigorously testing a new method under challenging conditions. We firmly believe that this testing and evaluation process holds significant value in establishing the proposed method as a viable alternative to the widely accepted cross-correlation method, which is considered the gold standard in tracking methods.
To the best of our knowledge, no previous study has conducted a similar test, making our research unique in its approach. The conclusions drawn from our work are particularly interesting and valuable, and we believe they contribute to the existing body of knowledge in this field. This aspect of our research can be seen as an innovation, as it fills a gap in the literature and provides new insights into the performance and potential applications of the proposed method.
Therefore, we contend that the reviewer's request for innovation has indeed been addressed through our thorough testing and the valuable conclusions drawn from our experimental work.
Moreover, the manuscript provides comprehensive details of the experimental setup, enabling the precise movement of objects at a scale of thousandths of pixels, as well as the necessary software for image processing. This level of detail allows for direct reproduction of the experiment, facilitating the calibration of the methods and potential implementation in other research applications. Although the setup and measuring conditions may appear simple due to years of testing and refinement, achieving such small displacements as we have accomplished is not commonly found in the existing literature.
Finally, we have successfully demonstrated that the Discrete Subtraction Method can serve as a complement or even a practical alternative to the cross-correlation method at a significantly lower computational cost. To the best of our knowledge, given the current level of development of tracking algorithms based on cross-correlation, proposing a working alternative holds substantial value..
On the other hand, in our understanding, the notion of "academic value" should not solely be confined to the introduction of new theories. Technical papers often face underappreciation and are disregarded due to perceived lack of innovation. However, we believe that the provision of experimental setups, calibration results, and validation findings establishes crucial foundations for the development of innovative applications, supported by a solid experimental and methodological background.
I hope this revision meets your expectations. If you have any further questions or need additional assistance, feel free to ask.
Some suggestions are as follows:
- Diagrams need to be added to help readers better understand the implementation of the methods.
We agree with this suggestion of the reviewer, and a new figure (Figure 3) has been added to illustrate the process.
- Principle analysis needs to be conducted to illustrate the results of the experiment, preferably mathematically.
The Introduction and Methods sections of the manuscript have been rewritten with the aim of providing clarity regarding the objectives of the study and the experimental procedures employed. Additionally, we acknowledge that certain results, particularly those presented in Table 1 and the findings from Figure 8, may be difficult to interpret. To address this concern, we have included additional explanations specifically targeting these results, aiming to enhance their understanding for the readers.
Reviewer 4 Report
1. It should be noted that the authors only examined the discrete subtraction and cross correlation, which are already available. Is there anything new that the authors can do here?
2.. In the abstract, it does not mention the reason why the authors chose to use object tracking in their study.
3. There should be a discussion of the statistical results in the conclusion along with suggestions for future research.
4. I would like to point out that the reference [3] is not cited, as well as all the references should be cited in the proper area.
5. It appears that the references are not in the correct order.
6. I would like to point out that the problem is not mentioned at the end of the introduction.
7. It is recommended that the last paragraph should be the conclusion of the introduction.
8. Figure 1 should be replaced by a block diagram instead of the figure.
9. In a manuscript that is about analysis, what is the purpose of Figure 2?
10. Can the authors tell me what the relation between [19] and the manuscript is?
11. Figure 3 shows the frame calculations, but the paper discusses object racking in the paper. Could you please clarify this? Why is Figure 3 necessary? What is its purpose?
12. There is a difference between the results obtained from regular shapes and the results the authors would like to get from these tracking methods
13. Figure 10 does not seem to be related to the manuscript in any way?
14. The conclusion is very long and needs to be rewritten only in order to clarify the objectives and future work.
15. There is a need to update the references.
Moderate editing of English language required
Author Response
Reviewer #4
- It should be noted that the authors only examined the discrete subtraction and cross correlation, which are already available. Is there anything new that the authors can do here?
The novelty of the manuscript relays in the precise calibration of the DSM method against the cross-correlation method, which is widely regarded as the gold standard in tracking methods. Having in mind the results obtained, we firmly believe that our contribution will be very useful for future development of tracking algorithms. Here we test a recent method under limit conditions. This method was presented as an alternative to the cross-correlation method enhanced with subpixel capabilities, which is computationally intensive and has some problems with the peak-locking effect.
Moreover, this manuscript provides comprehensive details of the experimental setup, enabling the precise movement of objects at a scale of thousandths of pixels, as well as the necessary software for image processing. This level of detail allows for direct reproduction of the experiment, facilitating the calibration of the methods and potential implementation in other research applications. Although the setup and measuring conditions may appear simple due to years of testing and refinement, achieving such small displacements as we have accomplished is not commonly found in the existing literature.
With this, we have successfully demonstrated that the Discrete Subtraction Method can serve as a complement or even a practical alternative to the cross-correlation method at a significantly lower computational cost. the best of our knowledge and given the current level of development of tracking algorithms based on cross-correlation, proposing a working alternative to cross-correlation that solves some of its problems, holds substantial value.
- In the abstract, it does not mention the reason why the authors chose to use object tracking in their study.
The Abstract has been rewritten to underline the purpose of our research.
- There should be a discussion of the statistical results in the conclusion along with suggestions for future research.
We have revised the text and added some comments in the Conclusion.
- I would like to point out that the reference [3] is not cited, as well as all the references should be cited in the proper area.
We are sorry with errors with references. Reference [3] was not cited since it was the same that former reference [21].
- It appears that the references are not in the correct order.
Aside of the error already commented, we did not find any other error. However, since new references have been added, all numbers have been re-checked to avoid additionoal errors.
- I would like to point out that the problem is not mentioned at the end of the introduction.
Thank you for the suggestion. The Introduction has been revised and written to include more details.
- It is recommended that the last paragraph should be the conclusion of the introduction.
As we said above The Introduction has been thoroughly revised and rewritten to clarify our aims and findings in the manuscript. We hope that you find it now correct.
- Figure 1 should be replaced by a block diagram instead of the figure.
We agree that a block diagram is needed. However, according to the text, we found that Figure 3 is a more convenient place for this chart.
- In a manuscript that is about analysis, what is the purpose of Figure 2?
The Methods section provides a comprehensive description of both the experimental setup and the methods employed. While the experiment details are thoroughly explained, we believe that including a picture of the real setup would greatly assist other researchers in replicating the experiment. It is worth noting that, to the best of our knowledge, there are only a few manuscripts that offer detailed instructions on how to achieve displacement tests with an accuracy of 0.002 px. Therefore, we strongly believe that providing as much information as possible in our manuscript will be beneficial to the scientific community.
- Can the authors tell me what the relation between [19] and the manuscript is?
Reference [19] serves as a link to the repository of our university, which contains the programs utilized in this manuscript. It is important to note that the current link is generic and does not provide access to the software at this time. We have taken precautions to ensure that the software is not uploaded until the manuscript has been accepted. We appreciate your understanding regarding this matter and assure you that the necessary software will be made available once the manuscript is accepted.
Anyway, the data and the programs are temporally available at https://drive.google.com/drive/folders/1ht-Im75ygkrzxQ4LXNWH_JkkPYfA1-9I?usp=drive_link
- Figure 3 shows the frame calculations, but the paper discusses object racking in the paper. Could you please clarify this? Why is Figure 3 necessary? What is its purpose?
We are sorry, but we do not understand very well your question, since former figure 3 is key to show the results we have found. We explain here in brief the data that figure (now figure 4):
In the context of our study, the targets are printed and affixed to a platform that is in motion. As the platform moves, a series of frames are captured, with each frame capturing the target at a specific position. In Figure 3 (now Figure 4), we present the calculated displacement of the target between successive frames. This visual representation, in conjunction with the data presented in Table 1, demonstrates the robustness of each method when tested under ideal conditions. Consequently, it establishes the maximum level of accuracy that can be expected from each method, considering the specific characteristics of the target being examined.
This explanation has been also added to the text.
- There is a difference between the results obtained from regular shapes and the results the authors would like to get from these tracking methods
Regular shapes are frequently encountered in various real-world applications. These shapes can either be printed and attached to a moving object or can be derived from existing object features such as bolts or holes. In certain cases, a defocused point can be interpreted as a Gaussian shape, and calibration of many 3D systems is performed using a checkerboard pattern. Therefore, despite the apparent simplicity of the analysis, the obtained results can be readily extrapolated to real-world experiments, thereby enhancing their practical applicability.
We have added some comments to this in the text, in order to clarify the aim of the manuscript and the results obtained.
- Figure 10 does not seem to be related to the manuscript in any way?
Figure 11 (formerly Figure 10) presents the error between the theoretical position of the target and its corresponding calculated position. Essentially, it serves as a complementary visualization to Figures 5 to 10 (formerly 4 to 9), which illustrates the total movement. While we may agree that Figure 11 is not strictly essential, it provides more detailed information regarding the results, making it potentially valuable for the reader's understanding and interpretation.
- The conclusion is very long and needs to be rewritten only in order to clarify the objectives and future work.
We can’t see the relevance on the conclusion section length. However, we have revised this section to make it clearer. Additionally, we have included some comments about future developments.
- There is a need to update the references.
As we explained before, all references have been revised and updated.
Round 2
Reviewer 2 Report
The authors have made some revisions, but unfortunately, the description and discussion of the innovation of the manuscript are still insufficient, to the extent that the paper is not sufficient to be accepted.
Minor editing of English language required
