Efficient Measurement Method: Development of a System Using Measurement Templates for an Orthodontic Measurement Project
Round 1
Reviewer 1 Report (Previous Reviewer 5)
Dear authors,
thank you for such submission. It is a very well developed paper, simple to read even for not software developers professional.
I think that if you address these minor comments it could be suitable for pubblication.
Lines 87-89: please modify in: “The aim of this study is to evaluate the efficiency of the system, while the accuracy of the measurement results will be left to the evaluation of the orthodontist user.”
Sections 2.3-2.4: as illustrated in fig. 2 and 3, the cephalograms that you talk about are not 2D slices of DICOM file, but the axial, coronal and sagittal images of 3D reconstruction, which is more simple to consider and may avoid the errors due to selection of correct 2D slice. Please specify that in material and methods.
Results section: may be interesting to consider the average time to process the measurement from the orthodontist. The five orthodontist probably completed the measurements in different time and it is interesting to have a mean time of elaboration of the single patients, that with a large sample of 500 patients could be very similar to clinical daily practice.
I wish congratulate with you for the work you've done.
Author Response
Response to Reviewer 1 Comments
Point 1: Lines 87-89: please modify in: “The aim of this study is to evaluate the efficiency of the system, while the accuracy of the measurement results will be left to the evaluation of the orthodontist user.”
Response 1: We have corrected it as you indicated.
The sentence "This paper has been written from the perspective of the system's developers." has been retained because of its importance.
Point 2: Sections 2.3-2.4: ’as illustrated in fig. 2 and 3, the cephalograms that you talk about are not 2D slices of DICOM file, but the axial, coronal and sagittal images of 3D reconstruction, which is more simple to consider and may avoid the errors due to selection of correct 2D slice. Please specify that in material and methods.
Response 2: In Section 2.3, the words "coronal" and "sagittal" should have been written as "anterior" and "lateral" rendering images. This is a simple mistake. This is now corrected.
Due to this mistake, the part in question could have been read as referring to slices, and the intent of the sentence might not have been conveyed correctly. Please check again about it.
Point 3: Results section: may be interesting to consider the average time to process the measurement from the orthodontist. The five orthodontist probably completed the measurements in different time and it is interesting to have a mean time of elaboration of the single patients, that with a large sample of 500 patients could be very similar to clinical daily practice.
Response 3: We were interested in this point, but we did not have the information at the time, and now it is difficult to confirm.
However, it is possible that a paper written by the Doctor has written about this point.
The system could also be modified to record the work time, for example, by setting up a mechanism to monitor the update time of files.
This is not mentioned in Section 4.4, but could be added.
Please see the attachment for details of the modifications.
Sincerely,
Koga
Author Response File: 
Author Response.pdf
Reviewer 2 Report (Previous Reviewer 4)
Dear authors, good luck for the next projects!
Author Response
Thank you for your review.
We will continue our efforts to have the paper published.
Sincerely,
Koga
Reviewer 3 Report (Previous Reviewer 2)
As I see it this is a resubmission of the article.
The authors have improved their mauscript, made changes that were suggested in earlier reviews.
In its current form, in my opinion, the article can be published in Software MDPI.
Author Response
Thank you for your review.
We will continue our efforts to have the paper published.
Sincerely,
Koga
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
The paper is very informative and take you in a step by step presentation but the introduction is very poor and needs more relative literature for the readers and the methods can be presented in a clearer way.
Author Response
Thank you for your review.
We believe you are correct in the points you raised, and we have examined the manuscript again.
Please find the revisions we have made as follows.
> the introduction is very poor and needs more relative literature for the readers
The contents of the Introduction section have been widely expanded.
See line numbers [21-89].
References have also been added.
Reference numbers [1,3,4,5,10] are newly added ones.
> the methods can be presented in a clearer way
The contents of the Material & Method section have been widely expanded.
See line numbers [90-341].
In addition, the section Prerequisites for Plotting Measurement Markers has been removed as it was deemed redundant. (Line numbers [116-140] in the manuscript before revision.)
In addition, some descriptions of the definition of measurement points that we believe need to be supplemented have been added.
Please refer to the "Definition of Landmarks to be Measured" section of the text, line numbers [283-319].
Author Response File: Author Response.pdf
Reviewer 2 Report
The article deals with a very interesting issue.
However, it is created in a slightly different form from the standards.
It would be worthwhile to expand the introduction a bit, moreover, to refer to more references. Please clarify the purpose of the paper.
The conclusions also need to be expanded.
Despite these minor remarks, I think that the article is very interesting and with minor corrections can be accepted for publication.
Author Response
Thank you for your review.
We believe you are correct in the points you raised, and we have examined the manuscript again.
Please find the revisions we have made as follows.
> It would be worthwhile to expand the introduction a bit, moreover, to refer to more references.
The contents of the Introduction section have been widely expanded.
See line numbers [21-89].
References have also been added.
Reference numbers [1,3,4,5,10] are newly added ones.
> Please clarify the purpose of the paper.
The contents of the introduction have been supplemented with a description of the state of the field in which this system is located and the need for this system.
Please refer to line numbers [22-67].
> The conclusions also need to be expanded.
The contents of the Conclusions section have been slightly expanded (line numbers [466-470]).
Author Response File: Author Response.pdf
Reviewer 3 Report
We do not have to make any particular suggestion to the authors and we will accept in good faith the effectiveness of the programme since the only people competent to judge it are the orthodontists who participated in the study. We would like to believe that this work will facilitate the work of the scientists.
Author Response
Thank you for your review.
We believe you are correct in the points you raised, and we have examined the manuscript again.
Please find the revisions we have made as follows.
Thank you for your assessment regarding the importance of this paper.
The revised manuscript more clearly demonstrates the importance of this system.
We hope you will take the time to read it.
Author Response File: Author Response.pdf
Reviewer 4 Report
Dear authors,
congratulations for the topic addressed and the results obtained, with great importance for the imaging evaluation of the cranio-facial structures.
To clarify some aspects, please specify the following:
- were the measurements only linear or did they also allow the evaluation of some angles?
-were certain specifications mentioned for making the cephalogram?
-were the points selected on the cephalogram specific to an analysis technique or were they chosen based on the specifics of the population or the doctor's preferences?
Author Response
Thank you for your review.
We believe you are correct in the points you raised, and we have examined the manuscript again.
Please find the revisions we have made as follows.
> were the measurements only linear or did they also allow the evaluation of some angles?
This system was also capable of measuring angles.
However, it was not evaluated because it was not needed in the actual measurement project and angle measurements were not performed.
This point has been added to the "Limts" section of the "Discussion" section.
See line numbers [442-445].
> were certain specifications mentioned for making the cephalogram?
Yes, I have received instructions from the project manager (Doctor) regarding the specifications for the cephalograms to be attached to the measurement template.
I have added this to the Definition of Landmarks to be Measured section of the text.
See line numbers [283-319].
> were the points selected on the cephalogram specific to an analysis technique or were they chosen based on the specifics of the population or the doctor's preferences?
The instructions from the project manager (Doctor) mentioned earlier included a list of points to be selected in the cephalogram.
Thus, it is the latter.
This point is also mentioned in the aforementioned additional section.
See line numbers [283-319].
Author Response File: Author Response.pdf
Reviewer 5 Report
Dear authors,
Despite is a study with a very large samples, it presents many weaknesses.
1. The introduction is inadequate, without bibliography reference and not described the background of this study.
2. figure 1 not visible and very confusing
3. figure 2 not found
4. in "Prerequisites for Plotting Measurement Markers,Measurement Process, Format and Utilization of Aggregated Results" sections there are some formatting problems.
Results are only a patient (on 500) example. It lacks of statistical analysis between these patients and their measurement by different orthodontists, useful to demonstrate this software predictability.
Discussion too are inadeguate, without references.
Conclusion are not supported by results.
You had a good project idea, but it is already present, for so long.
Here I reported you some studies as example of recent papers regarding this topic:
Farronato M, Maspero C, Abate A, Grippaudo C, Connelly ST, Tartaglia GM. 3D cephalometry on reduced FOV CBCT: skeletal class assessment through AF-BF on Frankfurt plane-validity and reliability through comparison with 2D measurements. Eur Radiol. 2020 Nov;30(11):6295-6302. doi: 10.1007/s00330-020-06905-7. Epub 2020 May 7. PMID: 32382843.
Bajaj K, Rathee P, Jain P, Panwar VR. Comparison of the Reliability of Anatomic Landmarks based on PA Cephalometric Radiographs and 3D CT Scans in Patients with Facial Asymmetry. Int J Clin Pediatr Dent. 2011 Sep-Dec;4(3):213-23. doi: 10.5005/jp-journals-10005-1112. Epub 2011 Apr 15. PMID: 27678229; PMCID: PMC5034081.
Ed-Dhahraouy M, Riri H, Ezzahmouly M, Bourzgui F, El Moutaoukkil A. A new methodology for automatic detection of reference points in 3D cephalometry: A pilot study. Int Orthod. 2018 Jun;16(2):328-337. doi: 10.1016/j.ortho.2018.03.013. Epub 2018 Apr 6. PMID: 29628421.
Reporting some results with a statistical analysis of your software effectiveness it could be interesting because it is a continue-growing field of orthodontic research.
Kind regards
Author Response
Thank you for your review.
We believe you are correct in the points you raised, and we have examined the manuscript again.
Please find the revisions we have made as follows.
> 1. The introduction is inadequate, without bibliography reference and not described the background of this study.
The contents of the Introduction section have been widely expanded.
See line numbers [21-89].
References have also been added.
Reference numbers [1,3,4,5,10] are newly added ones.
> 2. figure 1 not visible and very confusing
Fig1 has been reworked into a flowchart format.
> 3. figure 2 not found
The layout of the manuscript was broken, which may have made it difficult to find the figures.
The layout of the main text has been completely corrected this time, and we believe you will be able to find the figures.
> 4. in "Prerequisites for Plotting Measurement Markers,Measurement Process, Format and Utilization of Aggregated Results" sections there are some formatting problems.
In the previous submission, the layout of the listings was not correct.
We believe this was due to the fact that we had delegated the formatting of the manuscript to Editor Office and our intentions were not properly reflected in the manuscript.
All of the layouts you indicated have now been corrected.
> Discussion too are inadeguate, without references.
The contents of the Discussion section have been widely expanded.
See line numbers [404-465].
> Results are only a patient (on 500) example. It lacks of statistical analysis between these patients and their measurement by different orthodontists, useful to demonstrate this software predictability.
Perhaps, at the time of our last submission, our writing style was misleading with respect to the scope of this paper's interest.
This paper is not about analyzing the results of the measurements, but rather the methodology of the measurements.
The description of this point was unclear, so we have added it to the introduction chapter.
Please refer to line numbers [21-89].
This point may also be related to the following remarks:
> Conclusion are not supported by results.
If you understand that what we wanted to evaluate in this paper was not the measurement results themselves but the efficiency of the measurement process, you will agree that the results support the conclusion.
> Here I reported you some studies as example of recent papers regarding this topic:
We have reviewed all of the references you provided and discussed their similarities to this study.
We were unable to identify any papers, including the one you provided, in which measurements were made using exactly the same method as in this paper.
We believe this is the novelty of this paper.
On the other hand, in expanding the references cited, we found the paper you provided very helpful as an example of a conventional approach.
Therefore, we have added the reference of Bajaj (2011) as paper number [5] as a reference.
We are so thank you for your support.
Author Response File: Author Response.pdf
