Phenotyping Diabetes Mellitus on Aggregated Electronic Health Records from Disparate Health Systems

Round 1

Reviewer 1 Report

This paper presented the results of the validation of an algorithm for detecting MS cases in mixed databases. The paper was interesting as indeed the situation of mixed data sources is encountered more often nowadays. The paper was very clearly written, and flowcharts help a lot in understanding the methods. I would only suggest adding more international context, especially in the discussion section and a couple of comments in the document attached. 

Comments for author File: Comments.pdf

Author Response

The authors thank the Reviewer for the detailed review of our manuscript and for the most useful suggestions. We have incorporated all of the suggested changes. Please find our responses to the Reviewer's comments below:

1. Adding more relevant references

We have included more references in the Discussion section, relating to other studies that have developed and proposed algorithms for similar purposes.

2. All the areas where the reviewer had inserted comments on the original pdf submission have been addressed in the revised manuscript (relevant regions are highlighted in yellow for ease of review). This includes comments on removing selected portions from the Results that were more of explanations instead.

Thank you once again for your kind review.

Best wishes,

Sreemanee Raaj Dorajoo (on behalf of all authors)

Reviewer 2 Report

Thank you for your submission.  It is quite interesting, albeit not particularly novel.

The authors should include a completed checklist, using the latest version of the PROBAST tool - Robert F. Wolff et al., Annals of Internal Medicine 2019 170:1, 51-58).  (www.probast.org).

It seems that the diagnostic codes alone detect the great majority of patients with diabetes (Table 3).

I would have thought that metformin could be problematic (e.g. its use for polycystic ovary disease), and this illustrates the minor role that medicines play in the algorithm.

I don’t see the relevance of the repeated comments about having data from multiple institutions – the algorithm and its variables are very simple and readily available, so why does that matter at all?

Author Response

The authors wish to express our thanks to the reviewer for closely reading our submission and for proposing edits that certainly improve the quality of our submission. We have provided a point-by-point response to all the comments raised by the reviewer below in blue fonts. 

Thank you for your submission.  It is quite interesting, albeit not particularly novel.

The authors should include a completed checklist, using the latest version of the PROBAST tool - Robert F. Wolff et al., Annals of Internal Medicine 2019 170:1, 51-58).  (www.probast.org).

We thank the reviewer for pointing out this to us. We have now included a completed checklist for review as supplementary material. We have filled in the relevant sections of the PROBAST checklist with the objective information to facilitate assessment. However, assessment of bias and applicability of our model is best performed by independent parties (other than the authors) given that there is an element of judgement involved in assessing these aspects.

It seems that the diagnostic codes alone detect the great majority of patients with diabetes (Table 3).

I would have thought that metformin could be problematic (e.g. its use for polycystic ovary disease), and this illustrates the minor role that medicines play in the algorithm.

This is indeed true. Diagnosis codes alone captured 77.0% and 83.4% of patients with diabetes in our 2 validation cohorts which means 23.0% and 16.6% were wrongly identified as non-diabetic patients (false negatives). Misclassification can also arises from false positive classification (erroneous presence of diagnostic codes for diabetes in non-diabetic patients). These sources of errors combined may still be acceptable in certain circumstances if diagnosis codes alone are used as long as the sensitivity and specificity values are known to facilitate adjustments through  quantitative bias analyses.

The reviewer rightly raises an important point that medications too may lead to misclassification as not all diabetic medications are used solely for diabetes management. It is indeed for this reason that we included the presence of persistently elevated blood glucose levels as a secondary requirement for detecting patients with diabetes prior to applying the medication criteria.

I don’t see the relevance of the repeated comments about having data from multiple institutions – the algorithm and its variables are very simple and readily available, so why does that matter at all?

Thank you for this comment. We raised this point to highlight that we are using a flexible threshold to identify elevated blood glucose levels. Different institutions use slightly different laboratory testing equipment and have varying ranges of normal blood glucose levels. The laboratory test value criteria in our algorithm uses the institution specific cut-offs for determining elevated glucose levels which is something we’ve not observed in other algorithms possibly owing to the fact that they were developed using data from a single healthcare institution. This admittedly is a minor point and we have since toned down on our emphasis of this in the manuscript’s Discussion section.

Thank you. 

Best wishes,

Sreemanee Raaj Dorajoo (on behalf of all authors)

Reviewer 3 Report

My further comments and suggestions are the following:

 - In the manuscript, "gender" should be replaced by "sex", that is more correct. In effect, sex is usually categorized as female or male but there is variation in the biological attributes that comprise sex and how those attributes are expressed. Gender refers to the socially constructed roles, behaviours, expressions and identities of girls, women, boys, men, and gender diverse people.

 - English language needs to be carefully revised and typos corrected.

English language needs to be carefully revised and typos corrected.

Author Response

We thank the reviewer for highlighting the language issue in our paper. 

We are able to see only the comments relating to the language by the reviewer. To be clear, the below is the entirety of what we can see listed as comments from the reviewer, in italics: 

"My further comments and suggestions are the following:

 - In the manuscript, "gender" should be replaced by "sex", that is more correct. In effect, sex is usually categorized as female or male but there is variation in the biological attributes that comprise sex and how those attributes are expressed. Gender refers to the socially constructed roles, behaviours, expressions and identities of girls, women, boys, men, and gender diverse people.

 - English language needs to be carefully revised and typos corrected."

The authors are unsure if there were additional comments from the reviewer that may have been left out for some reason. If so, we are happy to respond again to any uncommunicated comments. 

With regards to the comment on language and grammar, we have since proof-read the manuscript again and made various grammatical corrections. The term 'gender' has also been changed to sex as suggested. 

We thank the reviewer once again for your time to read and suggest changes to our manuscript. 

Best wishes,

Sreemanee Raaj Dorajoo (on behalf of all authors)