Salt Reduction Using a Smartphone Application Based on an Artificial Intelligence System for Dietary Assessment in Patients with Chronic Kidney Disease: A Single-Center Retrospective Cohort Study
Round 1
Reviewer 1 Report
The authors present a study on the clinical utility of the App in patients with chronic kidney disease. The authors promise an evaluation of the usefulness of the App to monitor the intake of salt consumption.
1. Initially, the purpose and title of the paper is confusing. On the one hand, it is usefulness evaluation and on the other hand it is to measure salt intake using urine tests, and in addition, to perform digital processing of images of food dishes. Authors must reorganize the paper to focus and articulate the content of the manuscript.
2. The authors must present the design of the App in detail, scientifically measure the usability (ease or difficulty of use) according to the characterization of the sample of volunteers. It is well known that the digital divide is a factor of valuable analysis. Also, why is there a trial operation period?
3. The authors argue that the study is quasi-experimental. From my experience it is a descriptive study, reaching the analysis of the data. I cannot visualize what was the "intervention" or experiment that they carried out on the patients (volunteers), because they were only measured in a period of time.
4. The authors must clarify the evaluation variables of the utility of an App. They must specify the method and the scientific standard used.
5. How was the sample calculated?
6. The primary outcome was estimated as salt intake after 3 months of using the app and at a 6-month follow-up. About this, how did the App promote the short-term reduction of salt intake?
7. It is necessary to know why the application for smartphones, Gohan Coach, was chosen.
8. It is necessary to know details of the validated web application program interface (API). Development process and validation process.
9. What was the manual modification by users.
10. Finally, the authors must specify the code assigned by the Hospital's ethics committee and, if possible, attach it, in order to understand the endorsed protocol.
When reviewing the references, several are considered old for this type of study:
Ref 5 of 2004
Ref 6 of 2006
Ref 10 of 2022
Ref 19 of 2005
Ref 21 of 2001
Ref 23 and 27 of 2006
Among others more
Author Response
The authors present a study on the clinical utility of the App in patients with chronic kidney disease. The authors promise an evaluation of the usefulness of the App to monitor the intake of salt consumption.
- Initially, the purpose and title of the paper is confusing. On the one hand, it is usefulness evaluation and on the other hand it is to measure salt intake using urine tests, and in addition, to perform digital processing of images of food dishes. Authors must reorganize the paper to focus and articulate the content of the manuscript.
Response: We would like to thank the reviewer for this perceptive comment, and we apologize for our confusing title and original description of the purpose of our study. The primary aim of the present study was to evaluate reductions in salt intake following the use of an App using 24-h urine collection, as described in the abstract (Page 1, Lines 14–16) and in the introduction section of the revised manuscript (Page 2, Lines 63–65). To assess the utility of the App, we retrospectively compared longitudinal changes in estimated salt intake between patients who used the App and those who did not. We have clarified the purpose of the present study by replacing the phrase “Clinical Usefulness of” with “Salt Reduction using” in the title of the revised manuscript. In addition, we have renamed the “2.1. Study Population” section as “2.1. Study Design and Population” to highlight the design of the present study (Page 2, Lines 67–73). Further, we have moved the description of 24-h urine collection to the “Outcome Measures and Data Collection” section (Pages 2–3, Lines 109–125).
Moreover, Gohan Coach, which performs digital processing of images of food dishes based on artificial intelligence, has been referred to as “An Artificial Intelligence System for Dietary Assessment” (Sensors 2020; 20: 4283). Accordingly, we have replaced the phrase “an Artificial Intelligence-Powered Smartphone Application” with “a Smartphone Application based on an Artificial Intelligence System for Dietary Assessment” in the title of the revised manuscript.
- The authors must present the design of the App in detail, scientifically measure the usability (ease or difficulty of use) according to the characterization of the sample of volunteers. It is well known that the digital divide is a factor of valuable analysis. Also, why is there a trial operation period?
Response: We strongly agree that details regarding the design of the app from Software Engineering, including the Requirements, Planning, Analysis, Design, Implementation, Testing, Installation or deployment, would capture the interest of the readers. However, given that we are clinical nephrologists (not the developers of the app) and this app (Gohan Coach) is developed by Kirin Holdings (Tokyo, Japan), we cannot explain these details unfortunately. Our knowledge is limited to the function of the app and how to use the app in clinical settings, as described in the Intervention and Follow-Up section (pages 4–5, lines 140–171) and Figure 2. Moreover, some functions of the app, including the algorithm-based feedback, is patent pending and highly confidential. On the other hand, information regarding the main function of the app, namely the AI-based meal image analysis function based on the web API “Calorie Check API” (https://iot.sonynetwork.co.jp/service/caloriecheck/, in Japanese) developed by Sony Network Communications Inc. (Tokyo, Japan) and a food composition database of around 110,000 items (https://www.eatsmart.co.jp/service/, in Japanese) provided by Eat Smart, Inc. (Tokyo, Japan), can be found on the Internet and is also available for commercial use. Therefore, we ask the reviewer’s consideration regarding this issue.
Regarding usability of the App, we wholeheartedly agree with the reviewer that usability is one of the most important aspects of the smartphone app in clinical use. Although we did not quantitatively assess the usability of the app with scientifically validated tools, there were no questions or complaints regarding the usability of the app from the study participants, which may partly be attributable to the inclusion of relatively young patients with a mean age of approximately 60 years in the present study. We have clarified this point in the results section of the revised manuscript (Page 8, Lines 250–252). In addition, we admit that patient satisfaction with the app, trust, and implementation of the app’s recommendations, which are important aspects for analyzing the effects of the app, were not systematically obtained in the present study. We have provided a description of this limitation in the discussion section of the revised manuscript (Page 12, Lines 389–392).
A trial operation period with free usage of the app was provided to patients referred by a limited number of medical institutions, including our clinic, as Kirin Holdings (Tokyo, Japan) considered the app to still be in the validation stage during the present study and it was too early to distribute widely for a fee. We have clarified these points in the introduction section of the revised manuscript (Page 2, Lines 54–57).
- The authors argue that the study is quasi-experimental. From my experience it is a descriptive study, reaching the analysis of the data. I cannot visualize what was the "intervention" or experiment that they carried out on the patients (volunteers), because they were only measured in a period of time.
Response: We would like to thank the reviewer for this comment. We believe that this study is not a descriptive but rather an experimental study as we compared changes in parameters between patients who used the app (cases) and those who did not use the app (controls) using a retrospective design and statistical analysis. Although the follow-up period of three or six months was relatively short, previous well-designed trials evaluating the effectiveness of interventions to reduce patient salt intake had similar follow-up periods of three or six months (ref. nos. 15 and 17). However, as the phrase “a quasi-experimental” is somewhat confusing, we have replaced this phrase and instead used the term “a single-center retrospective cohort study” in the revised title and in the introduction (Page 2, Lines 52–53), and “Study Design and Population” sections of the revised manuscript (Page 2, Lines 70–73).
- The authors must clarify the evaluation variables of the utility of an App. They must specify the method and the scientific standard used.
Response: We would like to thank the reviewer for this perceptive comment. As described in the “Outcome measures” section (Page 2, Lines 105–108), the primary study outcome was the change in salt intake estimated by 24-h urine collection, whereas the secondary outcomes included urinary protein excretion, blood pressure, body weight, body mass index, and estimated glomerular filtration rate. Additionally, as described above, we have moved the description of 24-h urine collection to the “Outcome Measures and Data Collection” section (Pages 2–3, Lines 109–125). In addition, we admit that patient satisfaction with the app, trust, and implementation of the app’s recommendations, which are important aspects for analyzing the effects of the app, were not systematically obtained in the present study. We have provided a description of this limitation in the discussion section of the revised manuscript (Page 12, Lines 389–392).
- How was the sample calculated?
Response: We thank the reviewer for this helpful comment. As this was not a prospective study but rather a retrospective study, a definitive sample size calculation was not performed. Instead, we performed an a priori calculation and determined the statistical power of the study was 0.83 assuming 13 and 22 patients were enrolled in the app user and app nonuser groups, which was calculated with a two-tailed significance value of 0.05 and an effect size of 1.05. We have provided a description of this point in the “Statistical Analyses” section of the revised manuscript (Page 7, Lines 198–207). An effect size of 1.05 was calculated from the difference in salt intake between groups in a randomized controlled trial assessing the effectiveness of sodium restriction in patients with CKD [ref no. 15] and from the standard deviation of the estimated salt intake in the control and intervention groups of a Japanese cohort [ref. no. 7]. We have clarified these points in the “Statistical Analysis” section of the revised manuscript.
- The primary outcome was estimated as salt intake after 3 months of using the app and at a 6-month follow-up. About this, how did the App promote the short-term reduction of salt intake?
Response: We would like to thank the reviewer for this helpful comment. As many Japanese foods are seasoned with salt, soy sauce, or soybean paste, we believe that dietary sodium restriction is challenging for Japanese people who are not aware of their actual salt intake. A previous Japanese study reported that awareness of the need for salt restriction led to only a 1 g/day reduction in urinary salt excretion [ref. no. 5]. A separate Japanese study reported that the proportion of individuals achieving an average urinary salt excretion of less than 6 g/day was only around 10% [ref. no. 6]. Based on these findings, we previously described that “visualization” of salt intake may be useful in reducing salt intake in patients [ref. no. 7], and that this App may increase patient awareness of their daily estimated salt intake in a timely manner. In contrast, the impact of the App on salt reduction was not sustained at 6-month follow-up visits after the end of the 3-month period of App use, as reported in previous trials [ref. nos. 15 and 17]. Further studies are warranted to examine whether continued use of the App or transition to other interventions at the end of the period of App use lead to sustained salt intake reductions. We have clarified these points in the discussion section of the revised manuscript (Page 11, Lines 318–335).
- It is necessary to know why the application for smartphones, Gohan Coach, was chosen.
Response: We would like to thank the reviewer for this perceptive comment. As described above, Kirin Holdings (Tokyo, Japan) considered the app to still be in the validation stage and that it was too early to distribute widely for a fee. The trial operation period with free usage of the app was provided to patients referred by a limited number of medical institutions, including our clinic. In addition, although there is now an application called goFOODTM (Sensors 2020; 20: 4283), it does not support Japanese and Gohan Coach was the only application that supported Japanese during the study period. We have added a relevant reference (ref. no. 9) and have clarified these points in the introduction section of the revised manuscript (Page 2, Lines 52–63).
- It is necessary to know details of the validated web application program interface (API). Development process and validation process.
Response: We strongly agree that details regarding the design of the app from Software Engineering, including the Requirements, Planning, Analysis, Design, Implementation, Testing, Installation or deployment, would capture the interest of the readers. However, given that we are clinical nephrologists (not the developers of the app) and this app (Gohan Coach) is developed by Kirin Holdings (Tokyo, Japan), we cannot explain these details unfortunately. Our knowledge is limited to the function of the app and how to use the app in clinical settings, as described in the Intervention and Follow-Up section (pages 4–5, lines 140–171) and Figure 2. Moreover, some functions of the app, including the algorithm-based feedback, is patent pending and highly confidential. On the other hand, information regarding the main function of the app, namely the AI-based meal image analysis function based on the web API “Calorie Check API” (https://iot.sonynetwork.co.jp/service/caloriecheck/, in Japanese) developed by Sony Network Communications Inc. (Tokyo, Japan) and a food composition database of around 110,000 items (https://www.eatsmart.co.jp/service/, in Japanese) provided by Eat Smart, Inc. (Tokyo, Japan), can be found on the Internet and is also available for commercial use. Therefore, we ask the reviewer’s consideration regarding this issue.
- What was the manual modification by users.
Response: We would like to thank the reviewer for this comment, and we apologize for our confusing description in the original manuscript. Details of the “manual modification” were described as follows in the “Intervention and Follow-Up” section (Page 4, Lines 152–158:
“Considering that it is impossible (1) to accurately grasp the amount of food and (2) determine differences in seasoning from meal images alone, a function that enables patients to manually modify the information if necessary was added to the Gohan Coach.”
Accordingly, we have modified the description in the introduction section of the revised manuscript to “The accuracy of the analysis is further enhanced by manual modifications of the information, including the amounts and the kinds of food and seasonings consumed by the users” (Page 2, Lines 61–63).
- Finally, the authors must specify the code assigned by the Hospital's ethics committee and, if possible, attach it, in order to understand the endorsed protocol.
Response: We would like to thank the reviewer for this comment, and we apologize for not attaching the approval number provided by the Ethics Committee. We have clarified this point in the “Study Design and Population” section of the revised manuscript (Page 2, Lines 68–70). Although the Ethics Application Documents were recorded in Japanese, we can provide them if necessary as supplemental files.
When reviewing the references, several are considered old for this type of study:
Ref 5 of 2004
Ref 6 of 2006
Ref 10 of 2022
Ref 19 of 2005
Ref 21 of 2001
Ref 23 and 27 of 2006
Among others more
Response: We thank the reviewer for these comments. We believe most of these refer to the results of important studies and trials that have been cited for a long time. In particular, there are limited reported trials focusing on interventions to reduce salt intake. Accordingly, we have only replaced the original ref. 23 (revised to ref. no. 24) with the latest study focusing on seasonal variation in blood pressure in hemodialysis patients (Front Cardiovasc Med 2022; 9: 820483).
Reviewer 2 Report
Yanai A and co-authors evaluate the utility of a smartphone application to reduce the dietary salt intake of a group of patients with chronic kidney disease. The importance of diet and its salt content in reducing hypertension and the progression of chronic kidney disease is universally accepted. However, patients adherence to a low sodium and dietary regimen is poor. The idea of ​​ applying artificial intelligence to diet control is extremely interesting. However, this study shows that even with a smartphone application, adherence is reduced after a few months. The work is conducted on a few patients and only on young patients. It would be more helpful to have a diverse and perhaps older population such as the one in which chronic kidney disease is most prevalent. In this way, the difficulties could be better understood.
Author Response
Yanai A and co-authors evaluate the utility of a smartphone application to reduce the dietary salt intake of a group of patients with chronic kidney disease. The importance of diet and its salt content in reducing hypertension and the progression of chronic kidney disease is universally accepted. However, patients adherence to a low sodium and dietary regimen is poor. The idea of ​​ applying artificial intelligence to diet control is extremely interesting. However, this study shows that even with a smartphone application, adherence is reduced after a few months. The work is conducted on a few patients and only on young patients. It would be more helpful to have a diverse and perhaps older population such as the one in which chronic kidney disease is most prevalent. In this way, the difficulties could be better understood.
Response: We would like to thank the reviewer for these valuable comments, and we wholeheartedly agree that the “digital divide” is an issue that cannot be ignored in healthcare. Accordingly, it is important to consider human factors when assessing the utility of smartphone applications (JMIR Mhealth Uhealth 2021; 9: e24467). Specifically in this study, as the reviewer pointed out, the recruited patients were rather young and it may be more helpful for further studies to include more diverse populations that comprise more elderly patients. We have added a relevant reference (ref. no. 32) and have provided descriptions of these points in the discussion section of the revised manuscript (Page 12, Lines 373–380).
Reviewer 3 Report
This is very interesting study evaluating the clinical usefulness of an artificial intelligence-powered 13 smartphone application in reducing the daily salt intake of patients with chronic kidney disease. There are several important comments.
1. There are only 15 patients included in the use of Gohan Coach. The lack of statistical significance in Table 1 could be due to lack of power. Did the investigators performed power calculation?
2. The number of patients that provided data on 24 hours urine collection is very small. Insignificant findings could be due to lack of power
3. The salt estimation method is unclear. How we can assure the accuracy?
4. Number of antihypertensive medication should also be included.
5. Chronic kidney disease diagnosis needs to be clarify, what eGFR cuttoff? what staging? how the investigators identified? by formula vs ICD code?
Author Response
This is very interesting study evaluating the clinical usefulness of an artificial intelligence-powered 13 smartphone application in reducing the daily salt intake of patients with chronic kidney disease. There are several important comments.
- There are only 15 patients included in the use of Gohan Coach. The lack of statistical significance in Table 1 could be due to lack of power. Did the investigators performed power calculation?
Response: We would like to thank the reviewer for this perceptive comment, and we wholeheartedly agree that the lack of statistical significance between the results shown in Table 1 may be attributable to the small sample size of the present study, although we did not calculate the statistical power required to detect differences between the groups for each variable in Table 1. We have provided a description of this limitation in the discussion section of the revised manuscript (Page 12, Lines 383–385). In addition, the post hoc power of the unpaired t-test was calculated as 0.68 (insufficient as < 0.80). We have provided a description of this calculation in the results section of the revised manuscript (Page 9, Lines 263–267).
- The number of patients that provided data on 24 hours urine collection is very small. Insignificant findings could be due to lack of power
Response: We would like to thank the reviewer for this perceptive comment, and we wholeheartedly agree that insignificant findings could be due to the lack of power. As described above, in this study, post hoc power for the unpaired t-test to compare the 3-month change in estimated salt intake (the primary outcome) between the groups was calculated as 0.68, which is not high enough (< 0.80), and the risk of type II error was undeniable. We have clarified this point in the Discussion section as a limitation (Page 12, Lines 394–397).
- The salt estimation method is unclear. How we can assure the accuracy?
Response: We thank the reviewer for this comment. Although 24-h urine collection has been the gold standard for evaluating salt intake, it has some limitations including its complexity, difficulty in performing repeat assessments, and need for the assessment of 24-h creatinine excretion to ensure the validity of urine collection. In the present study, the accuracy of 24-hr urine collection was verified by each physician, although we could not strictly and prospectively ensure the accuracy of 24-h urinary collection. First, the start and end time points of urine collection were determined to confirm that the duration of urine collection was 24 h and that patients did not miss collecting any urine samples. Second, each physician ensured that variation in 24-h urinary creatinine excretion and deviation from the estimated 24-h creatinine excretion was less than 0.2 g/day [ref. nos. 11 and 12]. We have provided a description of these points in the “Outcome Measures and Data Collection” section of the revised manuscript (Pages 3–4, Lines 109–125).
- Number of antihypertensive medication should also be included.
Response: We would like to thank the reviewer for this helpful comment. We have added the number of antihypertensive medications to the data in Table 1.
- Chronic kidney disease diagnosis needs to be clarify, what eGFR cuttoff? what staging? how the investigators identified? by formula vs ICD code?
Response: We thank the reviewer for this helpful comment. In the present study, CKD was not defined according to ICD code instead as an eGFR < 60 mL/min/1.73m2) and/or markers of kidney damage, including proteinuria, urine sediment abnormalities (including hematuria), and structural abnormalities. We have provided a description of this point in the “Study Design and Population” section of the revised manuscript (Page 2, Lines 76–78). In addition, we have added information regarding CKD stages to the “Outcome Measures and Data Collection” section (Page 4, Lines 126–128) and Table 1 of the revised manuscript.
Round 2
Reviewer 1 Report
The authors have made all the observations and comments of my review.
Reviewer 2 Report
After reading the revised version of the manuscript and the answers to the reviewers comments I believe the authors responded exhaustively to alla the issues raised by the reviewers. The revision improved the manuscript and it can be accepted for publication
