The Reliability of ClinCheck^® Accuracy before and after Invisalign^® Treatment—A Multicenter Retrospective Study

Round 1

Reviewer 1 Report

Clear aligner treatment is a hot topic of current research. This article investigated whether there is a correlation between the current (achieved) outcome and the final (predicted) outcome, as well as the magnitude of the difference between the current (achieved) outcome and the final (predicted) outcome.

The sample size of this study was large. However, there are some shortcomings:

1.     In the materials and methods section, it is not clear whether all the data (overjet, interincisal angle, overbite, intercanine width, intermolar width, and crowding) were obtained from the treatment evaluation tool in iTero.

2.     In Table 1, it is unclear why overjet is marked with ***, while other indicators (interincisal angle, intercanine width, and intermolar width) are marked with **, even though the p-values for these indicators were all <0.001.

3.     "=<0.001" in the results section should be written as "<0.001."

4.     In the results, whether the correlation coefficient indicates a strong correlation or weak correlation is defined differently in different literature. Therefore, it is preferred to state and cite the relevant literature in the manuscript.

5.     The Pearson correlation analysis in the results can only indicate that there is a correlation between the current (achieved) outcome and the final (predicted) outcome. Correlation cannot demonstrate whether the difference between the current (achieved) outcome and the final (predicted) outcome is statistically significant (other statistical methods should be used to test whether the difference is statistically significant).

6.     In the results section, clinical significance should be added. Values that differed by >0.5 mm or 2° were considered clinically significant. ([1] Al-Nadawi M, Kravitz ND, Hansa I, Makki L, Ferguson DJ, Vaid NR. Effect of clear aligner wear protocol on the efficacy of tooth movement: A randomized clinical trial. The Angle Orthodontist 2020; 91:157-163. [2]Hansa I, Katyal V, Ferguson DJ, Vaid N. Outcomes of clear aligner treatment with and without Dental Monitoring: A retrospective cohort study. American Journal of Orthodontics and Dentofacial Orthopedics 2021; 159:453-459. [3] Kieser M, Hauschke D. Assessment of clinical relevance by considering point estimates and associated confidence intervals. Pharmaceutical Statistics 2005; 4:101-107.)

7.     In the discussion section, first paragraph, the authors claim that "this is the first study" is inappropriate since there are relevant literatures cited in the following paragraphs.

8.     In the discussion section, the last line of the fourth paragraph, the authors claim that "differences were not statistically significant". The term” statistically significant” is inappropriate without performing statistical tests.

9.     In the discussion section, paragraph 4th and 5th, direct comparison of accuracy (a percentage) between studies is not appropriate. When comparing a percentage, the size of the denominator (the amount of predicted movement) should be considered. For example, if the amount of predicted movement is 2mm, 50% is 1mm, while if the amount of predicted movement is 10mm, 50% is 5mm.

10.   In the discussion section, fifth paragraph, the authors claim that ”the conflicting results is due to the advanced material of G8 with better control as compared to the older generations”. The conclusion is not valid because the confounding factors (such as patient age, the amount of predicted movement, and staging) were not controlled for in the statistical process, and there was no corresponding control group. To control confounding factors, subgroup analysis or multiple factor analysis is needed.

Writing suggestions:

11.   The terminology used in the article is inconsistent. "Current" is used in the abstract and Table 1, "actual" is used in materials and methods, and "achieved" is used in the discussion. It is suggested to use the word "achieved" throughout the article.

12.   The terminology used in the article is inconsistent. In the sentence "using Pearson correlation (P < 0.05) to determine if predicted values were correlated with current values." in the abstract, "predicted" is used, while "final" is used in the results and Table 1. It is recommended to use the word "predicted" throughout the article.

13.   In the abstract, the sentence "However, there were no statistically significant differences between the centers." does not clarify what "differences" is. It is suggested to clarify that it is "accuracies".

14.   In the introduction section, the statement "5th generation of Invisalign and 8th generation of Invisalign" is inappropriate. Invisalign does not officially divide its products into multiple generations. "G" is not an acronym for any word, and G8 cannot be regard as the 8th generation. "G" only refers to the update and release of attachments and SmartForce features, which are additions or updates to the previous features.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

First of all, I thank the authors for the opportunity to review this manuscript regarding the accuracy of the Invisalign system between planned and achieved movement.

I briefly summarize the issues that should be investigated:

 

Abstracts:

- please modify in accordance with subsequent modifications

Introduction:

- include the aim of the study at the end of the introduction

- insert a paragraph that critically analyzes the measurement methods between the planned/obtained result, other than the data provided by Align technology

Materials and method:

- it is unclear how the measurements were made and by whom; for reasons of transparency it is necessary to indicate the construction of the measured distances because it is difficult to imagine some parameters on digital models, for example, the inter-incisal angle. The same thing with regard to crowding, it is not clear what calculation of dentoalveolar discrepancy was calculated.

- It would be useful to insert images showing the parameters evaluated.

Results:

- Insert the units of measure alongside the analyzed parameter without repeating them continuously in the table

- Specify the possible utility of showing the differences between the various clinics; if it is not there, eliminate table 2, especially if no statistical inference has been made.

Discussion:

I suggest reading and citing a paper that criticized the method used by Invisalign and Kravitz et al. for quantifying the accuracy of pre/post orthodontic movements.

https://pubmed.ncbi.nlm.nih.gov/35207290/

Add a paragraph to critically reevaluate the data obtained with this method because I believe that we cannot speak of accuracy if the way in which the teeth move in space is not defined for geometric and biomechanical rules.

Conclusions:

I think it is wrong to speak of an overall average between the measurements analyzed because very different parameters are confused; it could be reorganized differently (widths, crowding, inter-incision angle, etc).

References:

Edit in accordance with the journal guidelines

 

Good job but must be improved! 

Author Response

Please see the attachment. 

Author Response File: Author Response.docx

Round 2

Reviewer 1 Report

I thank you for the sincere responses from the authors to my issues.

There are some additional things that need attention.

 

1.The data in this article can provide more information if the statistical and clinical significance of the differences are calculated.

Sheet 1 of the attachment file (an excel file named Table 1) provides the method for calculating the statistical and clinical significance of the difference values, while Sheet 2 of the attachment file shows Table 1 after revision by me. According to the 95% confidence interval of the difference values, the difference values of overbite, upper intermolar width, upper crowding, and lower crowding are considered to be statistically and clinically significant. This means that these indicators require clinical attention.

(1) Please confirm that you have used one of the following formulas (without taking the absolute value) to calculate the difference value. If so, the calculation in Sheet1 is correct.

Formula 1: Difference value = Achieved value - Predicted value

Formula 2: Difference value = Predicted value - Achieved value

(2) The basis for determining statistical significance is "If the confidence interval does not include the value of zero effect, it can be assumed that there is a statistically significant result," with reference to [1] du Prel JB, Hommel G, Röhrig B, Blettner M. Confidence interval or p-value?: part 4 of a series on evaluation of scientific publications. Dtsch Arztebl Int 2009;106:335-9. This method is equivalent to the paired t-test.

(3) The basis for determining clinical significance is "(d) Large clinically significant effect. This situation occurs when the two-sided 95% confidence interval lies completely above the threshold," with reference to [2] Kieser M, Hauschke D. Assessment of clinical relevance by considering point estimates and associated confidence intervals. Pharm Stat 2005;4:101-7.

 

2. (1)"It is our belief that the conflicting results are due to the advanced material of G8 with better control as compared to the older generations." could be rewritten as “It is possible that the conflicting results are due to the updated aligner material SmartTrack (LE30) or updated SmartForce features of G8 as compared to the previous studies.”

（2）It is suggested to add the following sentences after “It is possible that the conflicting results are due to the updated aligner material SmartTrack (LE30) or updated SmartForce features of G8 as compared to the previous studies.”:

But it should be noted that multiple potential factors, such as the patient's age, the initial malocclusion, the treatment plan design, and patient compliance, can influence the accuracy of tooth movement, and it is insufficient to determine the clinical significance based on accuracy alone. Further studies are needed that use the difference between predicted and achieved values as the evaluation indicator and control for confounding factors.

Comments for author File: Comments.zip

Author Response

Dear Mr. Kyle Li,

 

We reiterate our gratitude for considering our manuscript and acknowledge the reviewer's valuable input. Our team has carefully incorporated the reviewer's suggestions and made appropriate modifications to the manuscript. To facilitate the review process, we have included a point-by-point response to the reviewer's feedback below. Please note that all changes have been highlighted in yellow in the revised manuscript.

 

We eagerly anticipate your response and thank you for your time and consideration.

 

Sincerely,

 

Fahad Alsulaimani

Corresponding Author

 

Point-by-Point Response:

 

 

1.The data in this article can provide more information if the statistical and clinical significance of the differences are calculated.

 

Sheet 1 of the attachment file (an excel file named Table 1) provides the method for calculating the statistical and clinical significance of the difference values, while Sheet 2 of the attachment file shows Table 1 after revision by me. According to the 95% confidence interval of the difference values, the difference values of overbite, upper intermolar width, upper crowding, and lower crowding are considered to be statistically and clinically significant. This means that these indicators require clinical attention.

 

(1) Please confirm that you have used one of the following formulas (without taking the absolute value) to calculate the difference value. If so, the calculation in Sheet1 is correct.

 

Formula 1: Difference value = Achieved value - Predicted value

 

Formula 2: Difference value = Predicted value - Achieved value

 

(2) The basis for determining statistical significance is "If the confidence interval does not include the value of zero effect, it can be assumed that there is a statistically significant result," with reference to [1] du Prel JB, Hommel G, Röhrig B, Blettner M. Confidence interval or p-value?: part 4 of a series on evaluation of scientific publications. Dtsch Arztebl Int 2009;106:335-9. This method is equivalent to the paired t-test.

 

(3) The basis for determining clinical significance is "(d) Large clinically significant effect. This situation occurs when the two-sided 95% confidence interval lies completely above the threshold," with reference to [2] Kieser M, Hauschke D. Assessment of clinical relevance by considering point estimates and associated confidence intervals. Pharm Stat 2005;4:101-7.

 

 Response 1: We would like to thank the reviewer for their insight. We would like to confirm that formula 1 and table 1 have been used and this has been reflected in the manuscript.

 

2. (1) "It is our belief that the conflicting results are due to the advanced material of G8 with better control as compared to the older generations." could be rewritten as “It is possible that the conflicting results are due to the updated aligner material SmartTrack (LE30) or updated SmartForce features of G8 as compared to the previous studies.”

 

（2) It is suggested to add the following sentences after “It is possible that the conflicting results are due to the updated aligner material SmartTrack (LE30) or updated SmartForce features of G8 as compared to the previous studies.”:

 

But it should be noted that multiple potential factors, such as the patient's age, the initial malocclusion, the treatment plan design, and patient compliance, can influence the accuracy of tooth movement, and it is insufficient to determine the clinical significance based on accuracy alone. Further studies are needed that use the difference between predicted and achieved values as the evaluation indicator and control for confounding factors.

 

Response 2: We would like to thank the reviewer for their valuable input. We have made the corrections highlighted in yellow.

Reviewer 2 Report

I would thank the authors for the modifications provided in the revised form of the manuscript.

Regards

Author Response

Dear Mr. Kyle Li,

We would like to express our sincere appreciation for considering our manuscript. We would also like to extend our gratitude to the reviewer for providing us with valuable input and for their prompt response.

Thank you very much.

Sincerely,

Fahad Alsulaimani

Corresponding Author