Ten Years (2011–2021) of the Italian Lombardy ADHD Register for the Diagnosis and Treatment of Children and Adolescents with ADHD

Round 1

Reviewer 1 Report

I have a few doubts and suggestions that should be addressed before publication. The main concerns are related to data interpretation and description. Overall, data are interesting and worth to be published but the way they are presented and discussed should be strongly improved.

1. Not all methods were described in the Method section (eg.: structure of the figure from the Continuity of Care and Management subtitle)
2. Tables should be more informative and self-descriptive - all abbreviations should be explained, all statistical methods should be named. It is not clear to what analysis OR or p level refer to, what is comparing?
3. In the results authors indicate the higher rate of comorbidities. Among them the higher rate of anxiety disorders and conduct disorder. But OR=0.70 and OR=0.49 suggest rather the lower rate of disorders in the ADHD group. Compare with data in the table with %. Authors should carefully check all the data and their interpretation again before the publication.
4. May authors add the analysis of changes of examined characteristics between 2011 and 2021?
5. Discussion should be expanded. In its recent form it does not present valuable interpretation of the data based on the comparison of the project results with other publications.
6. May authors provide the information on limitation of their approach?

Author Response

REVIEWER 1

I have a few doubts and suggestions that should be addressed before publication. The main concerns are related to data interpretation and description. Overall, data are interesting and worth to be published but the way they are presented and discussed should be strongly improved.

1. Not all methods were described in the Method section (eg.: structure of the figure from the Continuity of Care and Management subtitle)

Response: we added the description of the Radar Chart both in the methods and in the results.

1. Tables should be more informative and self-descriptive - all abbreviations should be explained, all statistical methods should be named. It is not clear to what analysis OR or p level refer to, what is comparing?

Response: all these information were reported in the data analysis, however we added some notes in the tables.

1. In the results authors indicate the higher rate of comorbidities. Among them the higher rate of anxiety disorders and conduct disorder. But OR=0.70 and OR=0.49 suggest rather the lower rate of disorders in the ADHD group. Compare with data in the table with %. Authors should carefully check all the data and their interpretation again before the publication.

Response: we corrected these data

1. May authors add the analysis of changes of examined characteristics between 2011 and 2021?

Response: the aim of the study was to analyse the global 10 years data, anyway the characteristics data between 2011 and 2021 would be too small to draw significant conclusions. 

1. Discussion should be expanded. In its recent form it does not present valuable interpretation of the data based on the comparison of the project results with other publications.

Response: the aim of the study was to describe the progression of the project and the characteristics of the patients added to the register. The unicity of the project doesn’t allow to compare the results with other publications, except for the ADHD patients general characteristics (e.g. gender distribution, diagnosis, comorbidities, treatments..).

1. May authors provide the information on limitation of their approach?

Response: we integrated the discussion with the limitation of the study

Reviewer 2 Report

„A review of patient medical records was the design of this study.“ This is not a study design. This is obviously a retrospective study based on medical records.

Figure 1: It does not make sense to compare absolute numbers in age distributions in samples with such imbalances. Histograms with relative frequencies have to be displayed for this purpose.

The stepwise selection process of the logistic regression has to be described in detail and justified with citations as this is very much criticised in the methodological literature.

What do the asterisks in Table 1 mean: <0.0001 *?

Multiple tests are performed which cause alpha inflation. Bonferroni correction has to be applied.

Table 1: Chi-square tests: effect size Cramér V has to be added and discussed as it is well-known that large sample sizes can produce spurious significant results. The same holds for t-tests (effect size Cohen´s d has to be added and discussed).

Table 1 is not comprehensible. OR is the OR resulting from the simple contingency table? Which variables remained as predictors in each logistic regression analysis? A pseudo-R-square (McFadden´s R-square) has to be calculated in order to assess model fit.

Table 2: Same comments as for Table 1. How did you perform logistic regressions on some of the scales (e.g. CGI-S)? Did you dichotimize into pathological/non-pathological? Criteria for this? This is unclear and has to be described in the Methods section. Apart from this, the applied scales have to be described in more detail (reliability, validity) in the Methods section.

Table 3: kappa values <.40 have to be interpreted as just fair. Therefore, kappa corrects by chance agreement and consequently agreement in % should not be mainly looked at.

Where are the explicit analyses refering to the 12-18 month follow-up period mentioned in the abstract?

Author Response

REVIEWER 2

“A review of patient medical records was the design of this study.“ This is not a study design. This is obviously a retrospective study based on medical records.

Response: we provided to modify the sentence

Figure 1: It does not make sense to compare absolute numbers in age distributions in samples with such imbalances. Histograms with relative frequencies have to be displayed for this purpose.

Response: Below the histogram of the F1.

The stepwise selection process of the logistic regression has to be described in detail and justified with citations as this is very much criticised in the methodological literature.

Response: Table 1 stepwise selection

Table 2 stepwise selection

What do the asterisks in Table 1 mean: <0.0001 *?

Response: * = p < 0,05

Multiple tests are performed which cause alpha inflation. Bonferroni correction has to be applied.

Response: we don’t make multiple comparisons tests. In Table 1 we compare ADHD yes and ADHD no. In Table 2 we compare pharmacological vs psychological. In Table 3 CPRS vs CTRS and CBLC mother vs CBCL father

Table 1: Chi-square tests: effect size Cramér V has to be added and discussed as it is well-known that large sample sizes can produce spurious significant results. The same holds for t-tests (effect size Cohen´s d has to be added and discussed).

Response: Below the Cramer V in the tables. For the continuous variables we used the Wilcoxon test, for this reason the Wilcoxon effect size was reported.

Cramer V

Modello di regressione logistica: diagnosiadhdid(ref='2') = classeetadiagnosiid sessoid figliounicoid natoitaliaid adottivoid gradoscuolaid ripetenteid sostegnoscolasticoidistruzionesuperioremadreid istruzionesuperiorepadreid occupazionemadreid occupazionepadreid adhdfamigliaid partocesareoid prematuroid sottopesoid ritardomotorioid ritardolinguisticoid comorbidita_app comorbidita_sonno comorbidita_opp comorbidita_ansia comorbidita_ritment comorbidita_umore comorbidita_ling comorbidita_tic comorbidita_cond comorbidita_autismo comorbidita_coord comorbidita_altro patologiecronicheid

To assess the model fit we calculated Test della bontà di adattamento di Hosmer e Lemeshow

Since p>0.05 there isn’t an evidence of lack of fit.

Table 2. Clinical Characteristics of the ADHD Patients by Treatment Prescription.

Modello di regressione logistica: prescrizionefV1id(ref='2') = sottotipoadhdid qipatologicoid cprsaid ctrsaid cprsbid ctrsbid cprscid ctrscid cprshid ctrshid cprsjid ctrsjid cgisid

comorbidita_app comorbidita_sonno comorbidita_opp comorbidita_ansia comorbidita_ritment comorbidita_umore comorbidita_ling comorbidita_tic comorbidita_cond

comorbidita_autismo comorbidita_coord comorbidita_altro

Since p>0.05 there isn’t an evidence of lack of fit.

Table 1 is not comprehensible. OR is the OR resulting from the simple contingency table? Which variables remained as predictors in each logistic regression analysis? A pseudo-R-square (McFadden´s R-square) has to be calculated in order to assess model fit.

Response:  OR is the OR resulting from the simple contingency table. As reported in the method, logistic is the results of the multivariate logistic regression analysis with stepwise selection

Modello di regressione logistica: diagnosiadhdid(ref='2') = classeetadiagnosiid sessoid figliounicoid natoitaliaid adottivoid gradoscuolaid ripetenteid sostegnoscolasticoidistruzionesuperioremadreid istruzionesuperiorepadreid occupazionemadreid occupazionepadreid adhdfamigliaid partocesareoid prematuroid sottopesoid ritardomotorioid ritardolinguisticoid comorbidita_app comorbidita_sonno comorbidita_opp comorbidita_ansia comorbidita_ritment comorbidita_umore comorbidita_ling comorbidita_tic comorbidita_cond comorbidita_autismo comorbidita_coord comorbidita_altro patologiecronicheid

To assess the model fit we calculated Test della bontà di adattamento
di Hosmer e Lemeshow

Since p>0.05, there isn’t an evidence of lack of fit.

Table 2: Same comments as for Table 1. How did you perform logistic regressions on some of the scales (e.g. CGI-S)? Did you dichotimize into pathological/non-pathological? Criteria for this? This is unclear and has to be described in the Methods section.

Response: we attached the results of the Cramer V; The Clinical Global Impression – Severity scale (CGI-S) is a 7-point scale that requires the clinician to rate the severity of the patient's illness at the time of assessment, relative to the clinician's past experience with patients who have the same diagnosis. As reported in the manual for psychopharmacology (Guy, W. (1976). ECDEU assessment manual for psychopharmacology. US Department of Health, Education, and Welfare, Public Health Service, Alcohol, Drug Abuse, and Mental Health Administration, National Institute of Mental Health, Psychopharmacology Research Branch, Division of Extramural Research Programs) the range are : CGIS 5-7 = pathological, 1-4 = normal range

Modello di regressione logistica: prescrizionefV1id(ref='2') = sottotipoadhdid qipatologicoid cprsaid ctrsaid cprsbid ctrsbid cprscid ctrscid cprshid ctrshid cprsjid ctrsjid cgisid

comorbidita_app comorbidita_sonno comorbidita_opp comorbidita_ansia comorbidita_ritment comorbidita_umore comorbidita_ling comorbidita_tic comorbidita_cond

comorbidita_autismo comorbidita_coord comorbidita_altro

To assess the model fit we calculated Test della bontà di adattamento
di Hosmer e Lemeshow

Since p>0.05, there isn’t an evidence of lack of fit.

Apart from this, the applied scales have to be described in more detail (reliability, validity) in the Methods section.

Response: we added the references

Table 3: kappa values <.40 have to be interpreted as just fair. Therefore, kappa corrects by chance agreement and consequently agreement in % should not be mainly looked at.

Response: we are agree with the comment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

One of my comments was:

1. Discussion should be expanded. In its recent form it does not present valuable interpretation of the data based on the comparison of the project results with other publications.

I don't agree with the response:

Response: the aim of the study was to describe the progression of the project and the characteristics of the patients added to the register. The unicity of the project doesn’t allow to compare the results with other publications, except for the ADHD patients general characteristics (e.g. gender distribution, diagnosis, comorbidities, treatments..).

The importance of the project is related to the possibility of making a comparisons between the regions of the country or with other countries. So, "patients general characteristics (e.g. gender distribution, diagnosis, comorbidities, treatments" should be discussed in details and compared with other studies  - this should be important part of this article and would increase the value of the study. Are these characteristics similar to the results of previous studies/ regions/countries? 

Author Response

1. Discussion should be expanded. In its recent form it does not present valuable interpretation of the data based on the comparison of the project results with other publications.

I don't agree with the response:

Response: the aim of the study was to describe the progression of the project and the characteristics of the patients added to the register. The unicity of the project doesn’t allow to compare the results with other publications, except for the ADHD patients general characteristics (e.g. gender distribution, diagnosis, comorbidities, treatments..).

The importance of the project is related to the possibility of making a comparisons between the regions of the country or with other countries. So, "patients general characteristics (e.g. gender distribution, diagnosis, comorbidities, treatments" should be discussed in details and compared with other studies  - this should be important part of this article and would increase the value of the study. Are these characteristics similar to the results of previous studies/ regions/countries? 

Response: The previous response was not corrected although comparisons with other studies were reported.

We apologize and thank the reviewer. In the current revision we have further expanded the discussion by agreeing with what you suggest.

Reviewer 2 Report

The histogram in Figure 1 is still based on absolute numbers. Relative frequencies have to be chosen, otherwise the two samples are not comparable in this variable due to the different sample sizes.

"Children were 9 years old" is too unspecific. "Children had a median age of 9 years" is the correct expression.

The applied questionnaires are still not described in detail as it is supposed to be in a scientific article (reliability, validity).

"we don’t make multiple comparisons tests". The authors of course perform multiple tests. In Table 3 eight p values are listed, obviously originating from 8 Chi-square tests. Therefore, Bonferroni correction has to be p=.05/8= .006. This also has to be done in Tables 1 and 2 and p values have to be interpreted accordingly.

The selection process of logistic regression (forward?, backward?) is still not adequately described and discussed.

Cramér-V and Wilcoxon effect sizes have been added but they have not been interpreted in the text. This is incorrect.

Table 3: The authors have not adapted their interpretation: “substantial” agreement between parents and teachers (>60%)" although kappa values signal just a fair agreement. This is not methodologically sound.

Table 3: Chi-square tests have to be added with effect size Cramér-V and this has to be interpreted due to possible spurious statistical significance due to large sample size.

Author Response

Commento 1

1. The histogram in Figure 1 is still based on absolute numbers. Relative frequencies have to be chosen, otherwise the two samples are not comparable in this variable due to the different sample sizes.

Response

Below the chart with the relative frequencies of the two sample

Commento 2

"Children were 9 years old" is too unspecific. "Children had a median age of 9 years" is the correct expression.

Response

We modified the text

Commento 3

The applied questionnaires are still not described in detail as it is supposed to be in a scientific article (reliability, validity).

Risposta

The rating scales cited are the most used and approved scales for the assessment of ADHD diagnosis worldwide. The diagnostic assessment pathway was agreed, approved and shared by all participating ADHD centres. For additional information about psychometrics properties, validity and reliability, we refer to

Goyette, C.H.; Conners, C.K.; Ulrich, R.F. Normative data on revised Conners’ Parent and Teacher Rating Scales. J. Abnorm. Child Psychol. 1978 6, 221-236

Conners, C.K.; Sitarenios, G.; Parker, J.D.; Epstein, J.N. The revised Conners’ Parent Rating Scale (CPRS-R): Factor structure, reliability, and criterion validity. J. Abnorm. Child Psychol. 1998, 26, 257-268

Achenbach, T. M.; Edelbrock C.S.. Manual for the child behavior checklist and revised child behavior profile. 1983, 85-121.

Guy, W. Clinical Global Impressions. ECDEU Assessment Manual for Psychopharmacology—Revised. 1976; pp. 218-222 Rockville, USA

Commento 4

"we don’t make multiple comparisons tests". The authors of course perform multiple tests. In Table 3 eight p values are listed, obviously originating from 8 Chi-square tests. Therefore, Bonferroni correction has to be p=.05/8= .006. This also has to be done in Tables 1 and 2 and p values have to be interpreted accordingly.

Risposta

These are bivariate analysis all done with independent variables, where we don’t make pairwise comparisons, so in our opinion, the Bonferroni correction for the p-value significance is not applicable.

Commento 5

The selection process of logistic regression (forward?, backward?) is still not adequately described and discussed.

Risposta

The two models are calculated with stepwise selection method: the SELECTION=STEPWISE option is similar to the SELECTION=FORWARD option except that effects already in the model do not necessarily remain. Effects are entered into and removed from the model in such a way that each forward selection step can be followed by one or more backward elimination steps. The stepwise selection process terminates if no further effect can be added to the model or if the current model is identical to a previously visited model.

These are the selection processes for the models in Tables 1 and 2:

Table 1 – Selection process

Table 1 – Type III effects analysis (significance of an effect with all the other effects in the model)

Table 1 – Hosmer-Lemeshow lack of fit test: p=0.0533

Table 2 – Selection process

Table 2 – Type III effects analysis (significance of an effect with all the other effects in the model)

Table 2 – Hosmer-Lemeshow lack of fit test: p=0.0635

Commento 6

Cramér-V and Wilcoxon effect sizes have been added but they have not been interpreted in the text. This is incorrect.

Risposta:

We added the interpretation in the methods in the text.

Commento 7

Table 3: The authors have not adapted their interpretation: “substantial” agreement between parents and teachers (>60%)" although kappa values signal just a fair agreement. This is not methodologically sound.

Risposta:

We adapted the interpretation.

Commento 8

Table 3: Chi-square tests have to be added with effect size Cramér-V and this has to be interpreted due to possible spurious statistical significance due to large sample size.

Risposta:

We added V Cramer effect size