Next Article in Journal
Progress in Research on the Alleviation of Glucose Metabolism Disorders in Type 2 Diabetes Using Cyclocarya paliurus
Next Article in Special Issue
COVID-19 and Virtual Nutrition: A Pilot Study of Integrating Digital Food Models for Interactive Portion Size Education
Previous Article in Journal
Mapping of Outdoor Food and Beverage Advertising around Spanish Schools
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Nutrients
Volume 14
Issue 15
10.3390/nu14153168
nutrients-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Validating Accuracy of an Internet-Based Application against USDA Computerized Nutrition Data System for Research on Essential Nutrients among Social-Ethnic Diets for the E-Health Era

by
Ya-Ling Yang
1,
Hsiao-Ling Yang
1,
Joyce D. Kusuma
2,3,* and
Shyang-Yun Pamela Koong Shiao
4,*
1
School of Nursing, College of Medicine, National Taiwan University, Taipei 10051, Taiwan
2
Heritage Victor Valley Medical Group, Big Bear Lake, CA 92315, USA
3
Mt San Antonio Gardens, Pomona, CA 91767, USA
4
Center for Biotechnology and Genomic Medicine, Medical College of Georgia, Augusta University, Augusta, GA 30912, USA
*
Authors to whom correspondence should be addressed.
Nutrients 2022, 14(15), 3168; https://doi.org/10.3390/nu14153168
Submission received: 22 June 2022 / Revised: 23 July 2022 / Accepted: 28 July 2022 / Published: 31 July 2022
(This article belongs to the Special Issue E-health: A Novel Approach for Dietary Interventions)
Browse Figures
Review Reports Versions Notes

Abstract

:
Internet-based applications (apps) are rapidly developing in the e-Health era to assess the dietary intake of essential macro-and micro-nutrients for precision nutrition. We, therefore, validated the accuracy of an internet-based app against the Nutrition Data System for Research (NDSR), assessing these essential nutrients among various social-ethnic diet types. The agreement between the two measures using intraclass correlation coefficients was good (0.85) for total calories, but moderate for caloric ranges outside of <1000 (0.75) and >2000 (0.57); and good (>0.75) for most macro- (average: 0.85) and micro-nutrients (average: 0.83) except cobalamin (0.73) and calcium (0.51). The app underestimated nutrients that are associated with protein and fat (protein: −5.82%, fat: −12.78%, vitamin B12: −13.59%, methionine: −8.76%, zinc: −12.49%), while overestimated nutrients that are associated with carbohydrate (fiber: 6.7%, B9: 9.06%). Using artificial intelligence analytics, we confirmed the factors that could contribute to the differences between the two measures for various essential nutrients, and they included caloric ranges; the differences between the two measures for carbohydrates, protein, and fat; and diet types. For total calories, as an example, the source factors that contributed to the differences between the two measures included caloric range (<1000 versus others), fat, and protein; for cobalamin: protein, American, and Japanese diets; and for folate: caloric range (<1000 versus others), carbohydrate, and Italian diet. In the e-Health era, the internet-based app has the capacity to enhance precision nutrition. By identifying and integrating the effects of potential contributing factors in the algorithm of output readings, the accuracy of new app measures could be improved.

1. Introduction

For personalized precision nutrition, adequate intakes of macro-and micro-nutrients could reduce disease risks of obesity, metabolic disease, cardiovascular disease, and cancer across an individual’s the life span [1,2,3,4]. Hence, accurate measurements of essential micro-nutrients, in addition to macro-nutrients, are vital in achieving precision nutrition [5,6]. Methyl donors including micro-nutrients such as vitamins (folate B9, cobalamin B12) and amino acids are crucial for DNA methylation in nutrigenomics pathways for personalized precision nutrition [7,8,9,10,11]. Various internet-based applications (apps) are rapidly developing in the e-Health era [12,13,14], that are accessible and convenient for assessing nutrient intake in real-time [15,16,17,18]. However, compared to established dietary measures, the accuracy of apps is yet to be examined for essential micro-nutrients in addition to macro-nutrients, with records of 1–3 day diaries and longer durations including a food frequency questionnaire (FFQ) [12,13,19].
The validity of four apps was examined against the established 1–3 day diaries of the Nutrition Data System for Research (NDSR), and the agreement using intraclass correlation coefficient (ICC) for macro-nutrients varied, ranging from excellent (0.9–1.0) for calories, and moderate to excellent for other nutrients (carbohydrate: 0.65–0.99; protein: 0.64–1.00; fat: 0.52–1.0; fiber: 0.52–0.99; cholesterol: 0.77–1.0; sodium: 0.69–0.9) [13,19]. Using Pearson’s correlation, five apps validated against NDSR presented similar findings ranging from excellent (0.89–0.94) for calories, and moderate to excellent for other nutrients (carbohydrate: 0.73–0.94; protein: 0.83–0.96; fat: 0.75–0.88; fiber: 0.74–0.92; cholesterol: 0.62–0.81; sodium: 0.60–77) [19]. The differences between the apps and NDSR ranged from −7.92 to 8.35 for calories [13], with an underestimation of 7–41% for protein, fat, fiber, cholesterol, and sodium by the apps [19]. However, prior validation studies did not include essential micro-nutrients, which are critical in nutrigenomics pathways for personalized nutrition across various populations consuming various diet types [13,20]. Thus, macro- and micro-nutrients need to be examined across various social-ethnic diets for the apps against the established method.
Cultural ethnic inheritance, social change, and individual preferences influence human dietary habits [21], where traditional ethnic diets include examples of convenient American, Mediterranean, Italian, Chinese, and Japanese diets [13,19,20,21,22,23,24]. Additionally, fast foods [25,26], high school foods [26], smoothies [27,28,29], pure liquids [27,30], and dozens of other diet types emerged following urbanization and growing aging populations. While Mediterranean and Japanese diets are beneficial to prevent cardiovascular disease; some Western diets are loaded with empty calories leading to metabolic disorders [31,32], and rice-based ethnic diets contain insufficient zinc and iron to meet the daily requirements [33]. While smoothies have been commonly loaded with vegetables and fruits, they might increase daily caloric intake via excess sugar and fat intake if unchecked, with insufficient essential nutrients [33,34]. Hence, we included pure liquids and smoothies [27,33,34,35] as they are common additions to social-ethnic human diets in various healthcare settings. Additionally, convenient diets [25,26] and various social-ethnic diets are common in a social-ethnic context, thus they were also included in this study [21,36,37].
The Bland-Altman analysis is a commonly used method to examine the validity of a new measure against another measure [13,19,38,39,40,41]. Additionally, machine-learning-based, artificial-intelligence (AI), generalized regression (GR) models could be employed to identify the source factors that could contribute to the differences between measures [7,42], with enhanced prediction accuracy [7,12]. A previous study noted that the source factors that could contribute to the differences between the two measures of an app and FFQ included caloric ranges; the difference between two measures for carbohydrates, protein, fat; and diet types [12]. Therefore, in this study, we further validated the app against NDSR, to identify factors that could contribute to the differences between two measures for essential nutrients including various diet types, integrating AI method to confirm the source factors to the differences between the two measures.

2. Materials and Methods

We evaluated the essential nutrients from 131 social-ethnic diets taken by various vulnerable populations, continued from our series of studies [7,12,42]. Under-reporting is a common human behavior in human studies in recording dietary intake [13,43,44]; thus, validation of dietary intake with model social-ethnic diets as recipes or menus can offer precise and controlled nutrient intake for individuals to follow and consume [12,45]. We examined four groups of diets, including: (1) pure liquids [27,30,46]; (2) convenient diets [25,26,34]; (3) ethnic diets of Western and Eastern origins [21,24]; and (4) smoothies-added [27,28,29]. We selected various diets to enhance the delivery of nutrients based on possible forms of liquids and solid foods. To add variations to the baseline daily recipes of these diets, excess calories, proteins, vegetables, fruits, and fats were included.
We evaluated essential nutrients of these diets based on the United States National Institute of Health’s dietary reference intake [47] using both NDSR and the app based on 3-day 24-h dietary diaries throughout the week. Macro-nutrients included total calories, carbohydrates, protein, total fat, saturated fat, cholesterol, and fiber [12]. Micro-nutrients included vitamins: thiamin B1, riboflavin B2, niacin B3, pyridoxine B6, folate B9, and cobalamin B12, A, C, D, E; amino acids: methionine, glycine [47]; choline; and minerals: zinc, calcium, magnesium, iron, and sodium. Foods that are rich in carbohydrates also commonly contained fiber, vitamins A, B9, and C. Whereas, foods that are meat-based are rich in protein, fat, saturated fat, cholesterol, vitamin B12, methionine, glycine, and zinc [47].

2.1. Social-Ethnic Diets

Details of these diets were included in a prior report [12] are summarized in the following sections. Liquids are necessary for human hydration and gut motility [27,28]. Pure liquid diets have been commonly recommended for frail elderly individuals and patients with terminal illness and gastrointestinal dysfunction following surgery [27,30]. Examples of liquid diets comprise: various liquids loaded with minerals, fruits, vegetables, and soups. Convenient diets in modern industrial societies include canned-foods, café diets, and fast foods [25,26]. The canned-foods are comprised of items such as soups, various vegetables, noodles, and fish. The café diets include foods offered at high schools, frequently were loaded with sodium and saturated fat [34] such as chicken wings, chicken tenders, grilled cheese sandwiches, mini cheeseburgers, pizza, cheese sticks, hot dogs, and corn dogs. The fast-foods also contain high fat, saturated fat, sodium, sugar, and empty calories [25,26,48], such as fried meats, fries, biscuits, sandwiches, wraps, and hash browns [12,25,26].
Various ethnic diets include Western influences from American, Mexican, Italian, and Mediterranean diets, and Eastern influences from Japanese, Chinese, and Korean diets [21,24,37]. The American diet embodies fried foods and salads [31]. The Mexican diet contains grain- and bean-based foods with meats and chili soups. The Italian diet includes grains and vegetables. The Mediterranean diet contains plenty of fruits, vegetables, whole grains, nuts, and olive oil, with some meat [37]. The Japanese diet includes seafoods, soups, and tempura [49]. The Chinese diet embodies meats, tofu, eggs, noodles, and rice [50]. The Korean diet includes rice, meats, mixed bowls of vegetables, and soups [21,24]. A smoothie-added diet could be used when people perform physically strenuous activities that require additional hydrations or nutrients. The smoothie-added diet includes a variety of fruits and vegetables added to the base diet, with ethnic-based ingredients and multi-grains [27,28,29,51].

2.2. Dietary Measures and Nutrient Intake

We analyzed 3-day dietary intake of nutrients from various social-ethnic diets using both NDSR and an app. We used NDSR software version 2015 based on published values in the USDA nutrient database (NDSR, Minnesota University, MN, USA; http://www.ncc.umn.edu/products/nutrients-nutrient-ratios-and-other-food-components/primary-energy-sources/ (assessed on 1 June 2022)), which was initiated in 1974 by the National Heart, Lung, and Blood Institute (NHLBI) [7]. NDSR was developed from 19,000 foods that embody an array of ethnic foods and common menu items for analysis of 178 nutrients [12,52,53,54]. NDSR is widely used for nutrient analysis based on the 24-h diary, recipes, and menus, across populations in many countries with different diet types [12,52,53,54]. NDSR provided detailed reports on the quantity of macro- and micro-nutrients [55]. The app was developed by a nutrigenomics company that specialized in the internet-based mobile app to assess daily dietary intake (GB HealthWatch, San Diego, CA, USA, https://healthwatch360.gbhealthwatch.com, accessed on 1 June 2021) [12,23,56]. The app, based on daily food logs, could extricate 30 essential nutrients. Data accuracy was confirmed by team members before analysis.

2.3. Data Analysis

We analyzed data with JMP version 15.0.0 statistical software [57,58,59,60] (SAS Institute Inc., Cary, NC). We evaluated the agreement and bias between the apps and NDSR. Means and standard deviations (SD) for nutrients [61] were calculated for both measures. The agreement between the two measures for the parameters were analyzed using ICCs and mean % differences, and bias with standard errors (SE). Pairwise correlations for ICCs (r) between the two measures presented the strengths of association (excellent: ≥0.9, good: 0.75 ≤ 0.90, moderate: 0.50 ≤ 0.75, poor: <0.50) [39,55,62]. The Bland–Altman plots presented differences with limits of agreement (LoAs: mean differences ± 2 SD) between the two measures [12,63,64], with good agreement at 95% or greater [65,66,67]. The alpha for the significance level for all analyses was set at 0.05.
The analytics were described in detail in earlier studies [7,12,42], and summarized in the following. We employed GR models with AI-based machine learning methods to identify the source of differences between the two measures by progressively incorporating related factors in the analysis [7,12,42]. The source factors that could contribute to the differences between the two measures included (1) caloric ranges of total calories (<1000, 1000–2000, or >2000); (2) effects of differences from carbohydrates, protein, fat; and (3) diet types [12]. JMP software provided logistic regression (LR) as the default model to explain the baseline dependent variables. Following LR, we selected the Elastic Net estimated method (Leave-One-Out (LOO) and Validation Column for confirmatory analysis to predict the accuracy with smaller misclassification rate for minimal prediction error by avoiding over-fitting [7,12,42,68]. Elastic Net models were employed for effective use in handling complex multiple domains in the datasets, balancing potential interactions [69]. Both LOO and AICc validation are effective for small sample sizes and handling multiple domains [7,59]. LOO is used to select source factors within domains of caloric ranges, macro-nutrients, and diet types [70,71]. We used AICc validation columns to confirm how well the model fits with unbiased best model prediction [70], with 80/20 randomized split for training and validation set for predictive modeling. The selection of the best model is based on AICc (lower score is fitter, more precise), misclassification (smaller is more accurate), and area under the Receiver Operating Characteristics (ROC) curve (AUC, higher coverage is better) [42,60]. The interaction profilers were used to visualize the significant interactions, if found could be included in the final model [70,71].

3. Results

3.1. Agreement and Difference: The Bland-Altman Method

We validated 131 diets using the Bland-Altman method to assess the agreement (ICCs, % differences) and bias (SE) between the app and NDSR per three caloric ranges (<1000, 1000–2000, and >2000) and selected essential nutrients (Table 1) using 5–10% differences as agreement criteria [62]. The ICCs between the two measures for total calories were good for all diets together (r = 0.85), but moderate and worst for outliers of >2000 caloric range (<1000 calories: 0.75, 1000–2000 calories: 0.76, >2000 calories: 0.57 (95% Confidence Interval: 0.14–0.82)). The ICCs between the two measures were good for most other nutrients (average for macro- 0.85 and 0.83 for micro-nutrients), but moderate for cobalamin (0.73) and calcium (0.51) (all p < 0.001).
Compared to NDSR, the app presented acceptable differences (<5%) on total calories with the caloric range of <1000 and 1000–2000, but underestimated >2000 calories (>10%). For macro-nutrients, the app underestimated total calories and protein (>5%), fat and saturated fat (>10%); overestimated fiber (>5%); but presented acceptable differences with carbohydrate and cholesterol (<5%). For micro-nutrients, the app underestimated methionine (>5%); vitamin B12, glycine, zinc, and sodium (>10%); overestimated vitamin B1 and B9 (>5%), as well as vitamin A (>10%); but presented acceptable differences with vitamin B2, B3, B6, C, D, E, choline, calcium, magnesium, and iron (<5%). Thus, we found good agreement between the two measures on some micro-nutrients but not protein-based micro-nutrients that are essential in personalized precision nutrition.
The bias (SE) increased with increased caloric ranges (1.26 for <1000 calories, 1.65 for 1000–2000 calories, and 6.06 for >2000 calories). The bias was greater (>2) for vitamins A, C, E, and calcium when compared to all other nutrients including fiber and sodium (ranges 1.14–1.83). Furthermore, we used Bland–Altman plots to visualize the differences and LoAs between the app and NDSR. We presented side-by-side comparison of correlations (left panel) and Bland–Altman plots (right panel) on exemplary macro-nutrients of calories (Figure 1a,b), fat (Figure 2a,b), protein (Figure S1a,b), carbohydrate (Figure S2a,b); and micro-nutrients of folate (Figure 3a,b) and cobalamin (Figure 4a,b). Correlation plots presented good agreement between the two measures on all nutrients as presented in these figures. Bland–Altman plots displayed a wide variability of differences for both macro- (calories, fat, protein, carbohydrate) and micro-nutrients (folate, cobalamin). Furthermore, Bland–Altman plots illustrated the underestimation by the app compared to NDSR on total calories (−5.2%, Figure 1b), fat (−12.78%, Figure 2b), protein (−5.82%, Figure S1b), and cobalamin (−13.59%, Figure 4b); but overestimation on carbohydrate (0.93%, Figure S2b) and folate intake (9.06%, Figure 3b).
Additionally, we assessed the differences between the app and NDSR for various nutrients in association with caloric ranges and diets (Table 2 and Table S1–S4). To continue findings in Table 1 for the underestimation of total calories with >2000 calories, the app underestimated total calories with canned food, Mediterranean, Chinese, and smoothie-added diets (>5%), and fast food (>10%); but presented acceptable differences for other diets (<5%). For carbohydrates, the app presented acceptable differences with <1000 calories (<5%), but overestimated with 1000–2000 calories (>5%) and underestimated with >2000 calories (>5%). Additionally, for carbohydrates, the app underestimated with smoothie-added diets (>5%); overestimated with canned food and high school (>5%), and Mexican diets (>10%); but presented acceptable differences (<5%) with other diets (pure liquid, fast food, Italian, Mediterranean, American, Japanese, Chinese, and Korean). For protein, the app underestimated with <1000 calories (>5%) and >2000 calories (>10%), but presented acceptable differences with 1000–2000 calories (<5%); and underestimated with diets including pure liquids, Mexican, and smoothie-added diets (>5%), Mediterranean, Chinese, and Korean diets (>10%); but presented with acceptable differences with other diets of canned-food, high school, fast-food, Italian, American, and Japanese (<5%). The app underestimated fat with all caloric ranges (>10%); with diets, including high school, Mexican, Italian, Mediterranean, and Japanese diets (>5%); as well as canned food, fast food, Chinese, Korean, and smoothie-added diets (>10%); but presented acceptable differences with pure liquids and American diets (<5%). For folate, the app presented overestimation with 1000–2000 calories (>5%), <1000 calories (>10%), and underestimation with >2000 calories (>5%); overestimation with Italian, Mediterranean, American, Japanese, and Korean diets (>5%); pure liquid, canned food, high school, fast food, and Mexican diets (>10%); but acceptable differences with Chinese and smoothie-added diets (<5%). For cobalamin, the app underestimated with all caloric ranges (>10%); underestimated canned food and Italian (>5%) diets; pure liquid, high school, Mexican, Mediterranean, Chinese, Korean, and smoothie-added diets (>10%); overestimation with Japanese (>5%) diet; but presented acceptable differences with American and fast-food diets (<5%). Additionally, the app underestimated nutrients that are associated with protein- and fat-based intake, including saturated fat, cholesterol, methionine, choline, glycine, vitamins B12 and E; but overestimated nutrients that are associated with carbohydrate-based intake, including fiber, and vitamins B9, A, and C (Tables S1–S4).

3.2. Predictive Modeling for the Difference between the Internet-Based App against NDSR: Generalized Regression Analysis

For predictive modeling, we progressively examined significant factors per domains of caloric ranges (coded as one of the three versus the other two categories for <1000, 1000–2000, and >2000), energy-producing macro-nutrients, and diets. We included the significant factors of all domain factors in the final combined model (Table S5 progression examples for total calories, Table S6 for folate, and Table S7 for cobalamin). For total calories, differences of <1000 over other caloric ranges, fat, and protein were significant contributing factors to the difference between the app and NDSR (misclassification 0.19, AICc 30.29, and AUC 0.89) (Table 3, baseline LR model on the left panel and GR model validation on the right panel). As an example, Figure 5 illustrates the AUC curve for total calories with 89% for both sensitivity and specificity for the accuracy of the selected model [71]. Through the progressive analysis, we observed a higher AICc and less precise model by including an additional factor, the Chinese diet, thus producing a less favorable model than the selected model (Table S5).
We selected folate and cobalamin as the most representative essential micro-nutrients in precision nutrition for more detailed presentation. Factors that contributed to the differences in folate between the two measures included caloric range (<1000 versus the other two categories), carbohydrate, and Italian diet (misclassification 0.30, AICc 38.5, and AUC 0.90) (Table 4). Through the progression analysis, we noted a higher AICc and AUC by including an additional factor, the Chinese diet, thus, this factor was not included (Table S6). For cobalamin, significant factors that contributed to the differences between the two measures included protein, and American and Japanese diets (misclassification 0.26, AICc 37.7, and AUC 0.81) (Table 5). In the progression analysis, we observed a higher AICc by including additional factors of Chinese and Mexican diets, which presented less favorable models than the selected model; thus, they were not included in the final model (Table S7).
We also assessed significant factors between the app and NDSR for other nutrients in the progressive analysis, and summarized the final models in the Supplementary Tables (carbohydrate, protein, fat, saturated fat, cholesterol, and fiber in Table S8; thiamin, riboflavin, niacin, pyridoxine, choline, glycine, and zinc in Table S9; and vitamins A, C, D, and E, and calcium, magnesium, iron, and sodium in Table S10). Significant factors that contributed to the differences between the two measures for carbohydrate included calories, fiber, and Italian diet; for protein: caloric range (1000–2000 versus others), total calories, cholesterol, and canned foods; for fat: saturated fat, cholesterol, and fast food; for saturated fat: total calories, fat, and Korean diets; for cholesterol: fat and pure liquids; for fiber: caloric range of 1000–2000, carbohydrate, and canned-foods (misclassification 0.11–0.22, AICc 24.0–33.7, AUC 0.82–0.98). Similarly, significant factors for thiamin included total calories, fiber, high school, and Chinese diet; for riboflavin: carbohydrate, protein, and high school diets; for niacin: total calories and Chinese diet; for pyridoxine: total calories and canned-food; for choline: total calories, and canned-food; for glycine: protein, canned-food, and Japanese diet; and for zinc: total calories, protein, canned-food, and Japanese diet (misclassification 0.04–0.22, AICc 19.8–35.7, AUC 0.88–0.97). Additionally, significant factors for vitamin A included protein, and Korean and smoothie-added diets; for vitamin D: cholesterol, canned-food, and Mediterranean diet; for vitamin E: fat, fast food, and Italian diet; vitamin C: fiber, fast food, and Mexican diet; for calcium: cholesterol, and Mexican and Chinese diets; for magnesium: total calories, protein, fiber, canned-food, and high school food; for iron: protein, high school food and Mediterranean diet; for sodium: saturated fat, high school food, and Mexican diet (misclassification 0.11–0.33, AICc 31.2–41.7, AUC 0.73–0.90). The canned-food diet type was a common contributing factor to the differences between the two measures for protein, fiber, pyridoxine, choline, glycine, zinc, vitamin D, and magnesium (Tables S8–S10). We did not explicitly test the model for methionine, as methionine is purely dependent on protein for calculating the intake level. The interaction profiler plots did not present any significant three-way interactions for inclusion in the final models for all nutrients.

4. Discussion

For precision nutrition, we validated the accuracy of an internet-based app in assessing macro- and micro-nutrients in various social-ethnic diets, against 3-day NDSR in this study, in addition to previous validation against FFQ [13,14,19]. Compared to previous findings with excellent ICCs on calories (0.9–1) between the app and NDSR [13,19], we found good agreement (≥0.75) between the two measures for total calories (0.85). We further identified good agreement (≥0.75) between the two measures for total calories with caloric ranges of 1000–2000 (0.76), but moderate agreement with caloric ranges of <1000 (0.75) and >2000 (0.57). Compared to previous findings of moderate to excellent agreement for other major nutrients [13,19], the ICCs in this study between the two measures for most macro-nutrients were good (≥0.75) (average: 0.85 for all macro-nutrients). We further demonstrated the good agreement between the two measures using ICCs for most micro-nutrients (average 0.83), but found only moderate agreement for cobalamin (0.73) and calcium (0.51) (all p < 0.001).
In comparison to NDSR, the app underestimated protein and fat-based nutrients (protein: −5.82%, fat: −12.78%, vitamin B12: −13.59%, methionine: −8.76%, zinc: −12.49%), while it overestimated carbohydrate-based nutrients (fiber: 6.7%, B9: 9.06%). Thus, we found similar underestimation of protein and fat for macro-nutrients between the app and NDSR, as in a previous study [19]. Contrary to the validation against FFQ with the acceptable estimation of vitamins B1 (2.46%) and B9 (3.24%), this study demonstrated that the app overestimated vitamins B1 (6.48%) and B9 (9.06%) against NDSR. For choline, the app presented acceptable estimation (−4.51%) using 5% criteria, against NDSR, while it was underestimated with the app against FFQ (−6.23%). The bias (SE) was small for all three caloric ranges (0.63–0.88), between the two measures of the app and NDSR; which are smaller and more precise in this study compared to the findings in a prior study with the app against FFQ (1.44–5.91) [12,19]. The smaller bias could be due the same recording duration of 3-day diaries with both the measures in this study, as contrary to a longer duration in measure with FFQ for the prior study. The correlations between the two measures in this study were strong ≥0.70 for most nutrients (average for macro- 0.85 and 0.82 for micro-nutrients) except for calcium (0.51), that are of similar findings as in a prior study with app being validated against FFQ [12].
We identified the factors that contributed to the differences between the two measures. For total calories, the sources that contributed to the difference between the two measures included caloric range (<1000 versus others), fat, and protein. For folate, the sources included caloric range (<1000 versus others), carbohydrate, and Italian diet. Additionally, for cobalamin, the sources included protein, and American and Japanese diets. Therefore, continuing from a previous study [12], caloric ranges and various diets could be used to identify the sources that might contribute to the differences between the two measures, when validating a new measure such as an internet-based app against established dietary measures.
In summary, fat and protein continued to be the major nutrient sources of differences in the validation of the app against established dietary measures, when compared to NDSR as well as FFQ [12]. The confirmatory predictive modeling further substantiated this result, with specific caloric ranges and diet types also as sources of differences. Hence, these source factors could be used to adjust the algorithm of readings in the development of the internet-based app. With advanced technology and AI analytics in the e-Health era, these apps have the capacity to enhance precision nutrition, by integrating potential contributing factors in the development of new accurate measures. Various diets across populations and related factors should be included in future studies to further validate dietary measures.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/nu14153168/s1, Supplementary Table S1: Bias and agreement between internet-based applications and Nutrition Data System for Research per domains of caloric ranges, energy nutrients, and various diets for major nutrients (n = 131). Supplementary Table S2. Bias and agreement between internet-based application and Nutrition Data System for Research per domains of caloric ranges, energy nutrients, and various diets for vitamins Bs (n = 131). Supplementary Table S3. Bias and agreement between internet-based application and Nutrition Data System for Research per domains of caloric ranges, energy nutrients, and various diets for methyl donors and vitamins A, C, and D (n = 131). Supplementary Table S4. Bias and agreement between internet-based application and Nutrition Data System for Research per domains of caloric ranges, energy nutrients, and various diets for vitamin E and minerals (n = 131). Supplementary Table S5. Progression on selecting significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on total calories. Supplementary Table S6. Progression on selecting significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on folate. Supplementary Table S7. Progression on selecting significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on cobalamin. Supplementary Table S8. Summary of significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on major nutrients. Supplementary Table S9. Summary of significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on methyl-donors and co-factors. Supplementary Table S10. Summary of significant factors contributing to the differences between internet-based application and Nutrition Data System for Research on other vitamins and minerals. Supplementary Figure S1: (a) Correlation, (b) Bland and Altman plots between internet-based application and Nutrition Data System for Research (NDSR) for protein (a,b). Supplementary Figure S2. (a) Correlation, (b) Bland-Altman plots between internet-based applications and Nutrition Data System for Research (NDSR) for carbohydrate (a,b).

Author Contributions

Conceptualization and methodology: S.-Y.P.K.S.; acquisition and search of the literature: Y.-L.Y., H.-L.Y., J.D.K. and S.-Y.P.K.S.; data entry and verification of data accuracy Y.-L.Y., J.D.K., S.-Y.P.K.S. and H.-L.Y.; analysis and interpretation of data: S.-Y.P.K.S., J.D.K. and Y.-L.Y.; writing—original draft preparation: S.-Y.P.K.S. and J.D.K.; writing—review and editing: S.-Y.P.K.S., Y.-L.Y., J.D.K. and H.-L.Y. All authors have read and agreed to the published version of the manuscript.

Funding

Funding supports include the Doctoral Research Council Grants, Azusa Pacific University; Research Start-up fund from Augusta University awarded to the corresponding author (S.-Y.P.K.S.).

Institutional Review Board Statement

The parent study was approved by the appropriate Human Subjects Institutional Review Boards (IRB) from the California State Committee for the Protection of Human Subjects for data access through the CCR (CPHS-12-12-1007, approved 2013–2019), and from the local educational institutions (Azusa Pacific University, approved 2013–2015; Augusta University, 806069-7, approved 2015–2018).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the parent study.

Acknowledgments

The authors thank the support from GB HealthWatch (San Diego, CA, USA) on providing Internet-based App software. The authors acknowledge the contribution and assistance from Zenab Khan for coding and entering data.

Conflicts of Interest

The authors declare no potential conflict of interest concerning the research, authorship, and/or publication of this article.

References

  1. Bruins, M.J.; Van Dael, P.; Eggersdorfer, M. The role of nutrients in reducing the risk for noncommunicable diseases during aging. Nutrients 2019, 11, 85. [Google Scholar] [CrossRef] [PubMed]
  2. Kwon, Y.-J.; Lee, H.S.; Park, J.-Y.; Lee, J.-W. Associating Intake Proportion of Carbohydrate, Fat, and Protein with All-Cause Mortality in Korean Adults. Nutrients 2020, 12, 3208. [Google Scholar] [CrossRef] [PubMed]
  3. Cappuccilli, M.; Bergamini, C.; Giacomelli, F.A.; Cianciolo, G.; Donati, G.; Conte, D.; Natali, T.; La Manna, G.; Capelli, I. Vitamin B Supplementation and Nutritional Intake of Methyl Donors in Patients with Chronic Kidney Disease: A Critical Review of the Impact on Epigenetic Machinery. Nutrients 2020, 12, 1234. [Google Scholar] [CrossRef] [PubMed]
  4. Zeng, J.; Gu, Y.; Fu, H.; Liu, C.; Zou, Y.; Chang, H. Association Between One-carbon Metabolism-related Vitamins and Risk of Breast Cancer: A Systematic Review and Meta-analysis of Prospective Studies. Clin. Breast Cancer 2020, 20, e469–e480. [Google Scholar] [CrossRef] [PubMed]
  5. Kohlmeier, M.; De Caterina, R.; Ferguson, L.R.; Görman, U.; Allayee, H.; Prasad, C.; Kang, J.X.; Nicoletti, C.F.; Martinez, J.A. Guide and Position of the International Society of Nutrigenetics/Nutrigenomics on Personalized Nutrition: Part 2—Ethics, Challenges and Endeavors of Precision Nutrition. Lifestyle Genom. 2016, 9, 28–46. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Cunha, C.A.S.; Duarte, R.P. Multi-Device Nutrition Control. Sensors 2022, 22, 2617. [Google Scholar] [CrossRef]
  7. Shiao, S.P.K.; Grayson, J.; Lie, A.; Yu, C.H. Personalized nutrition—genes, diet, and related interactive parameters as predictors of cancer in multiethnic colorectal cancer families. Nutrients 2018, 10, 795. [Google Scholar] [CrossRef] [Green Version]
  8. Myte, R.; Gylling, B.; Schneede, J.; Ueland, P.M.; Häggström, J.; Hultdin, J.; Hallmans, G.; Johansson, I.; Palmqvist, R.; Van Guelpen, B. Components of one-carbon metabolism other than folate and colorectal cancer risk. Epidemiology 2016, 27, 787–796. [Google Scholar] [CrossRef] [Green Version]
  9. Niculescu, M.D.; Zeisel, S.H. Diet, methyl donors and DNA methylation: Interactions between dietary folate, methionine and choline. J. Nutr. 2002, 132, 2333S–2335S. [Google Scholar] [CrossRef] [PubMed]
  10. Amenyah, S.D.; Hughes, C.F.; Ward, M.; Rosborough, S.; Deane, J.; Thursby, S.-J.; Walsh, C.P.; Kok, D.E.; Strain, J.J.; McNulty, H.; et al. Influence of nutrients involved in one-carbon metabolism on DNA methylation in adults—A systematic review and meta-analysis. Nutr. Rev. 2020, 78, 647–666. [Google Scholar] [CrossRef]
  11. Anderson, O.S.; Sant, K.E.; Dolinoy, D.C. Nutrition and epigenetics: An interplay of dietary methyl donors, one-carbon metabolism and DNA methylation. J. Nutr. Biochem. 2012, 23, 853–859. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Kusuma, J.D.; Yang, H.-L.; Yang, Y.-L.; Chen, Z.-F.; Shiao, S.-Y.P.K. Validating Accuracy of a Mobile Application against Food Frequency Questionnaire on Key Nutrients with Modern Diets for mHealth Era. Nutrients 2022, 14, 537. [Google Scholar] [CrossRef] [PubMed]
  13. Lin, A.W.; Morgan, N.; Ward, D.; Tangney, C.; Alshurafa, N.; Van Horn, L.; Spring, B. Comparative Validity of Mostly Unprocessed and Minimally Processed Food Items Differs Among Popular Commercial Nutrition Apps Compared with a Research Food Database. J. Acad. Nutr. Diet. 2021, 122, 825–832.e1. [Google Scholar] [CrossRef]
  14. Shinozaki, N.; Murakami, K. Evaluation of the Ability of Diet-Tracking Mobile Applications to Estimate Energy and Nutrient Intake in Japan. Nutrients 2020, 12, 3327. [Google Scholar] [CrossRef] [PubMed]
  15. Verma, M.; Hontecillas, R.; Tubau-Juni, N.; Abedi, V.; Bassaganya-Riera, J. Challenges in Personalized Nutrition and Health. Front. Nutr. 2018, 5, 117. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. McMahon, E.J.; Jaenke, R.; Brimblecombe, J. A Mobile App to Rapidly Appraise the In-Store Food Environment: Reliability, Utility, and Construct Validity Study. JMIR mHealth uHealth 2020, 8, e16971. [Google Scholar] [CrossRef] [PubMed]
  17. Celis-Morales, C.; Livingstone, K.M.; Marsaux, C.F.; Macready, A.L.; Fallaize, R.; O’Donovan, C.B.; Woolhead, C.; Forster, H.; Walsh, M.C.; Navas-Carretero, S.; et al. Effect of personalized nutrition on health-related behaviour change: Evidence from the Food4Me European randomized controlled trial. Int. J. Epidemiol. 2016, 46, 578–588. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Titcomb, T.J.; Bisht, B.; Moore, D.D.; Chhonker, Y.S.; Murry, D.J.; Snetselaar, L.G.; Wahls, T.L. Eating Pattern and Nutritional Risks among People with Multiple Sclerosis Following a Modified Paleolithic Diet. Nutrients 2020, 12, 1844. [Google Scholar] [CrossRef] [PubMed]
  19. Griffiths, C.; Harnack, L.; Pereira, M. Assessment of the accuracy of nutrient calculations of five popular nutrition tracking applications. Public Health Nutr. 2018, 21, 1495–1502. [Google Scholar] [CrossRef] [Green Version]
  20. Ferrara, G.; Kim, J.; Lin, S.; Hua, J.; Seto, E. A Focused Review of Smartphone Diet-Tracking Apps: Usability, Functionality, Coherence With Behavior Change Theory, and Comparative Validity of Nutrient Intake and Energy Estimates. JMIR mHealth uHealth 2019, 7, e9232. [Google Scholar] [CrossRef]
  21. Waldfogel, J. Dining out as cultural trade. J. Cult. Econ. 2020, 44, 309–338. [Google Scholar] [CrossRef]
  22. Chen, J.; Berkman, W.; Bardouh, M.; Ng, C.Y.K.; Allman-Farinelli, M. The use of a food logging app in the naturalistic setting fails to provide accurate measurements of nutrients and poses usability challenges. Nutrition 2019, 57, 208–216. [Google Scholar] [CrossRef] [PubMed]
  23. Fallaize, R.; Franco, R.Z.; Pasang, J.; Hwang, F.; Lovegrove, J.A. Popular Nutrition-Related Mobile Apps: An Agreement Assessment Against a UK Reference Method. JMIR mHealth uHealth 2019, 7, e9838. [Google Scholar] [CrossRef] [PubMed]
  24. Medicine LibreTexts. Diets around the World. 2020. Available online: https://med.libretexts.org/@go/page/21470 (accessed on 1 April 2022).
  25. Kupka, R.; Siekmans, K.; Beal, T. The diets of children: Overview of available data for children and adolescents. Glob. Food Secur. 2020, 27, 100442. [Google Scholar] [CrossRef]
  26. Li, L.; Sun, N.; Zhang, L.; Xu, G.; Liu, J.; Hu, J.; Zhang, Z.; Lou, J.; Deng, H.; Shen, Z.; et al. Fast food consumption among young adolescents aged 12–15 years in 54 low- and middle-income countries. Glob. Health Action 2020, 13, 1795438. [Google Scholar] [CrossRef]
  27. Abikusno, N. Importance of nutrition and lifestyle for elderly during the COVID-19 pandemic. J. Biomedika Dan Kesehat. 2020, 3, 109–111. [Google Scholar] [CrossRef]
  28. Batchelor-Murphy, M.K.; Steinberg, F.M.; Young, H.M. Dietary and feeding modifications for older adults. AJN Am. J. Nurs. 2019, 119, 49–57. [Google Scholar] [CrossRef]
  29. Stoker, D.; Durward, C. Smoothies-Helpful or Harmful? Available online: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2856&context=extension_curall (accessed on 1 April 2022).
  30. Gellrich, N.-C.; Handschel, J.; Holtmann, H.; Krüskemper, G. Oral cancer malnutrition impacts weight and quality of life. Nutrients 2015, 7, 2145–2160. [Google Scholar] [CrossRef] [PubMed]
  31. Brown, W.V.; Carson, J.A.S.; Johnson, R.K.; Kris-Etherton, P. JCL Roundtable: Fast Food and the American Diet. J. Clin. Lipidol. 2015, 9, 3–10. [Google Scholar] [CrossRef]
  32. Sprietsma, J. Modern diets and diseases: NO–zinc balance. Med. Hypotheses 1999, 53, 6–16. [Google Scholar] [CrossRef] [PubMed]
  33. Zhang, C.-M.; Zhao, W.-Y.; Gao, A.X.; Su, T.-T.; Wang, Y.-K.; Zhang, Y.-Q.; Zhou, X.-B.; He, X.-H. How could agronomic biofortification of rice be an alternative strategy with higher cost-effectiveness for human iron and zinc deficiency in China? Food Nutr. Bull. 2018, 39, 246–259. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Liu, J.; Micha, R.; Li, Y.; Mozaffarian, D. Trends in food sources and diet quality among US children and adults, 2003–2018. JAMA Netw. Open 2021, 4, e215262. [Google Scholar] [CrossRef] [PubMed]
  35. Abdollahi, S.; Soltani, S.; de Souza, R.J.; Forbes, S.C.; Toupchian, O.; Salehi-Abargouei, A. Associations between Maternal Dietary Patterns and Perinatal Outcomes: A Systematic Review and Meta-Analysis of Cohort Studies. Adv. Nutr. 2021, 12, 1332–1352. [Google Scholar] [CrossRef] [PubMed]
  36. Mozaffarian, D. Diets from around the world—quality not quantity. Lancet 2011, 378, 759. [Google Scholar] [CrossRef]
  37. Felando, M.; Rasmussen, H.; Karmally, W.; Guyton, J.R. JCL roundtable: Healthy ethnic diets. J. Clin. Lipidol. 2020, 14, 274–281. [Google Scholar] [CrossRef]
  38. Evenepoel, C.; Clevers, E.; Deroover, L.; Van Loo, W.; Matthys, C.; Verbeke, K. Accuracy of Nutrient Calculations Using the Consumer-Focused Online App MyFitnessPal: Validation Study. J. Med. Internet Res. 2020, 22, e18237. [Google Scholar] [CrossRef]
  39. Saravia, L.; Miguel-Berges, M.L.; Iglesia, I.; Nascimento-Ferreira, M.V.; Perdomo, G.; Bove, I.; Slater, B.; Moreno, L.A. Relative validity of FFQ to assess food items, energy, macronutrient and micronutrient intake in children and adolescents: A systematic review with meta-analysis. Br. J. Nutr. 2020, 125, 792–818. [Google Scholar] [CrossRef]
  40. Sierra-Ruelas, É.; Bernal-Orozco, M.F.; Macedo-Ojeda, G.; Márquez-Sandoval, Y.F.; Altamirano-Martínez, M.B.; Vizmanos, B. Validation of semiquantitative FFQ administered to adults: A systematic review. Public Health Nutr. 2020, 24, 3399–3418. [Google Scholar] [CrossRef]
  41. Tosi, M.; Radice, D.; Carioni, G.; Vecchiati, T.; Fiori, F.; Parpinel, M.; Gnagnarella, P. Accuracy of applications to monitor food intake: Evaluation by comparison with 3-d food diary. Nutrition 2021, 84, 111018. [Google Scholar] [CrossRef]
  42. Shiao, S.P.K.; Grayson, J.; Lie, A.; Yu, C.H. Predictors of the healthy eating index and glycemic index in multi-ethnic colorectal cancer families. Nutrients 2018, 10, 674. [Google Scholar] [CrossRef] [Green Version]
  43. Chmurzynska, A.; Mlodzik-Czyzewska, M.A.; Malinowska, A.M.; Czarnocinska, J.; Wiebe, D.J. Use of a Smartphone Application Can Improve Assessment of High-Fat Food Consumption in Overweight Individuals. Nutrients 2018, 10, 1692. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  44. Ravelli, M.N.; Schoeller, D.A. Traditional Self-Reported Dietary Instruments Are Prone to Inaccuracies and New Approaches Are Needed. Front. Nutr. 2020, 7, 90. [Google Scholar] [CrossRef] [PubMed]
  45. Zhang, L.; Nawijn, E.; Boshuizen, H.; Ocké, M. Evaluation of the Recipe Function in Popular Dietary Smartphone Applications, with Emphasize on Features Relevant for Nutrition Assessment in Large-Scale Studies. Nutrients 2019, 11, 200. [Google Scholar] [CrossRef] [Green Version]
  46. Warren, J.; Bhalla, V.; Cresci, G. Postoperative Diet Advancement: Surgical Dogma vs. Evidence-Based Medicine. Nutr. Clin. Pract. 2011, 26, 115–125. [Google Scholar] [CrossRef]
  47. National Institute of Health. Nutrient Recommendations: Dietary Reference Intakes (DRI). Available online: https://ods.od.nih.gov/Health_Information/Dietary_Reference_Intakes.aspx (accessed on 1 April 2022).
  48. Palos Lucio, A.G.; Sansores Martínez, D.N.; Olvera Miranda, C.; Quezada Méndez, L.; Tolentino-Mayo, L. Nutritional Quality of Fast Food Kids Meals and Their Contribution to the Diets of School-Aged Children. Nutrients 2020, 12, 612. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Saito, A.; Okada, E.; Matsumoto, M.; Takimoto, H. Impact of updated standard tables of food composition on nutrient intakes in Japan. J. Food Compos. Anal. 2019, 79, 5–11. [Google Scholar] [CrossRef]
  50. Whitton, C.; Rebello, S.A.; Lee, J.; Tai, E.S.; van Dam, R.M. A Healthy Asian A Posteriori Dietary Pattern Correlates with A Priori Dietary Patterns and Is Associated with Cardiovascular Disease Risk Factors in a Multiethnic Asian Population. J. Nutr. 2018, 148, 616–623. [Google Scholar] [CrossRef] [PubMed]
  51. Salomonsen, S.J.C.; Berland, A.; Bentsen, S.B. High-energy smoothies for patients in nursing homes. Nor. J. Clin. Nurs./Sykepl. Forsk. 2017, 12, e-62986. [Google Scholar] [CrossRef]
  52. Marchetti, B.S.; Steluti, J.; Fisberg, R.M.; Marchioni, D.M.L. Performance of the Brazilian version of GloboDiet software for dietary intake assessment. Nutrire 2018, 43, 13. [Google Scholar] [CrossRef]
  53. Mehta, S.; Ruth Dugas, L.; Choo-Kang, C.; Bovet, P.; Forrester, T.; Bedu-Addo, K.; Lambert, E.V.; Plange-Rhule, J.; Riesen, W.; Korte, W.; et al. Consumption of Monounsaturated Fatty Acids Is Associated with Improved Cardiometabolic Outcomes in Four African-Origin Populations Spanning the Epidemiologic Transition. Nutrients 2021, 13, 2442. [Google Scholar] [CrossRef]
  54. Yip, P.S.P.; Chan, T.F.J.; He, Z.; Leung, L.K.; Leung, S.F.S.; Chen, Z.-Y. Quantification of breast milk trans fatty acids and trans fat intake by Hong Kong lactating women. Eur. J. Clin. Nutr. 2020, 74, 765–774. [Google Scholar] [CrossRef] [PubMed]
  55. Harmouche-Karaki, M.; Mahfouz, M.; Obeyd, J.; Salameh, P.; Mahfouz, Y.; Helou, K. Development and validation of a quantitative food frequency questionnaire to assess dietary intake among Lebanese adults. Nutr. J. 2020, 19, 65. [Google Scholar] [CrossRef] [PubMed]
  56. GB Health Watch. Available online: https://www.gbhealthwatch.com/ (accessed on 1 June 2022).
  57. Nutrition Coordianting Center. Nutrition Data System for Research. Available online: http://www.ncc.umn.edu/food-and-nutrient-database/ (accessed on 1 April 2022).
  58. Grayson, J.; Gardner, S.; Stephens, M. Building Better Models with JMP® Pro; SAS Press: Cary, NC, USA, 2015. [Google Scholar]
  59. Klimberg, R.; McCullough, B. Fundamentals of Predictive Analytics with JMP, 2nd ed.; SAS Press: Cary, NC, USA, 2016. [Google Scholar]
  60. SAS Institute. Overview of the Generalized Regression Personality. Available online: https://www.jmp.com/support/help/en/16.0/index.shtml#page/jmp/overview-of-the-generalized-regression-personality.shtml (accessed on 20 June 2022).
  61. National Coalition for Health Professional Education in Genetics. Family History Educational Aids. NCHPEG. Available online: http://www.nchpeg.org/index.php?option=com_content&view=article&id=145&Itemid=64 (accessed on 10 October 2021).
  62. Bzikowska-Jura, A.; Sobieraj, P.; Raciborski, F. Low Comparability of Nutrition-Related Mobile Apps against the Polish Reference Method—A Validity Study. Nutrients 2021, 13, 2868. [Google Scholar] [CrossRef] [PubMed]
  63. Beck, K.L.; Houston, Z.L.; McNaughton, S.A.; Kruger, R. Development and evaluation of a food frequency questionnaire to assess nutrient intakes of adult women in New Zealand. Nutr. Diet. 2020, 77, 253–259. [Google Scholar] [CrossRef] [Green Version]
  64. Lin, P.-I.; Bromage, S.; Mostofa, M.; Allen, J.; Oken, E.; Kile, M.L.; Christiani, D.C. Validation of a dish-based semiquantitative food questionnaire in rural Bangladesh. Nutrients 2017, 9, 49. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Erkkilä, A.T.; Järvinen, R.; Karvonen, H.; Keronen, L.; Tuppurainen, M.T. Validation of a semi-quantitative FFQ using food records as a reference in older women in the Kuopio Fracture Prevention Study (OSTPRE-FPS). Public Health Nutr. 2012, 15, 635–639. [Google Scholar] [CrossRef]
  66. Pakseresht, M.; Sharma, S. Validation of a quantitative food frequency questionnaire for Inuit population in Nunavut, Canada. J. Hum. Nutr. Diet. 2010, 23, 67–74. [Google Scholar] [CrossRef]
  67. Kowalkowska, J.; Slowinska, M.A.; Slowinski, D.; Dlugosz, A.; Niedzwiedzka, E.; Wadolowska, L. Comparison of a full food-frequency questionnaire with the three-day unweighted food records in young Polish adult women: Implications for dietary assessment. Nutrients 2013, 5, 2747–2776. [Google Scholar] [CrossRef] [Green Version]
  68. Wu, Y. Elastic net for Cox’s proportional hazards model with a solution path algorithm. Stat. Sin. 2012, 22, 271–294. [Google Scholar] [CrossRef] [Green Version]
  69. Shiao, S.P.K.; Grayson, J.; Yu, C.H.; Wasek, B.; Bottiglieri, T. Gene Environment Interactions and Predictors of Colorectal Cancer in Family-Based, Multi-Ethnic Groups. J. Pers. Med. 2018, 8, 10. [Google Scholar] [CrossRef] [Green Version]
  70. Cheng, H.; Garrick, D.J.; Fernando, R.L. Efficient strategies for leave-one-out cross validation for genomic best linear unbiased prediction. J. Anim. Sci. Biotechnol. 2017, 8, 38. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  71. SAS Institute. Fitting Linear Models, 2nd ed.; SAS Institute Inc.: Cary, NC, USA, 2017. [Google Scholar]
Nutrients 14 03168 g001 550
Figure 1. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on total calories.
Figure 1. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on total calories.
Nutrients 14 03168 g001
Nutrients 14 03168 g002 550
Figure 2. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on fat.
Figure 2. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on fat.
Nutrients 14 03168 g002
Nutrients 14 03168 g003 550
Figure 3. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on folate.
Figure 3. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on folate.
Nutrients 14 03168 g003
Nutrients 14 03168 g004 550
Figure 4. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on cobalamin.
Figure 4. (a) Correlation, (b) Bland-Altman plots between the internet-based application and Nutrition Data System for Research (NDSR) on cobalamin.
Nutrients 14 03168 g004
Nutrients 14 03168 g005 550
Figure 5. Predicting accuracy of total calorie analyses using internet-based application against NDSR: Area under the receiver operating characteristic curve (AUC) for baseline logistic regression model (a) and Elastic Net with validation model (b).
Figure 5. Predicting accuracy of total calorie analyses using internet-based application against NDSR: Area under the receiver operating characteristic curve (AUC) for baseline logistic regression model (a) and Elastic Net with validation model (b).
Nutrients 14 03168 g005
Table
Table 1. Agreement and bias for the internet-based application (App) against Nutrition Data System for Research (n = 131).
Table 1. Agreement and bias for the internet-based application (App) against Nutrition Data System for Research (n = 131).
Parametersr (95% CI) **% Difference
M ± SD		NDSR
M ± SD		App
M ± SD		SE±2 SD%
Calories (kcal)0.85 (0.80, 0.89)−5.20 ** ± 13.861333 ± 891.81215 ± 853.61.2195.42
      <1000 (n = 63)0.75 (0.62, 0.84)−4.26 ** ± 10.01806.5 ± 147.1756.6 ± 162.51.2695.65
      1000–2000 (n = 50)0.76 (0.60, 0.85)−4.10 * ± 11.671330 ± 302.91255 ± 346.31.6592.31
      >2000 (n = 18)0.57 (0.14, 0.82)−11.56 * ± 25.713183 ± 10422708 ± 14106.0695.00
Carbohydrate (g)0.85 (0.80, 0.89)0.93 ± 16.22180.60 ± 141.0178.4 ± 136.11.4292.37
Protein (g)0.85 (0.80, 0.89)−5.82 ** ± 13.0552.20 ± 33.2547.35 ± 33.101.1493.89
Fat (g)0.81 (0.75, 0.86)−12.78 ** ± 16.7347.88 ± 33.5637.76 ± 29.571.4693.13
Sat Fat (g)0.84 (0.79, 0.89)−13.77 ** ± 16.7014.26 ± 9.8111.54 ± 9.351.4694.66
Cholesterol (mg)0.88 (0.84, 0.91)−4.86 ** ± 15.57184.2 ± 117.3175.4 ± 121.71.3693.89
Fiber (g)0.85 (0.80, 0.89)6.70 ** ± 20.9317.99 ± 19.3018.46 ± 18.911.8392.37
Thiamin (mg)0.85 (0.80, 0.89)6.48 ** ± 14.001.04 ± 0.711.12 ± 0.731.2294.66
Riboflavin (mg)0.86 (0.81, 0.90)0.19 ± 13.721.26 ± 0.851.23 ± 0.811.2093.89
Niacin (mg)0.86 (0.80, 0.90)0.51 ± 14.1514.43 ± 9.5214.34 ± 10.401.2493.13
Pyridoxine (mg)0.86 (0.80, 0.90)−4.38 ** ± 15.171.74 ± 1.461.60 ± 1.421.3394.66
Folate (mcg)0.87 (0.81, 0.90)9.06 ** ± 18.74280.2 ± 219.6302.3 ± 215.91.6492.37
Cobalamin (mcg)0.73 (0.64, 0.80)−13.59 ** ± 18.573.42 ± 2.052.70 ± 1.561.6292.37
Methionine (g)0.84 (0.79, 0.89)−8.76 ** ± 13.201.17 ± 0.721.03 ± 0.741.1593.89
Choline (mg)0.82 (0.76, 0.87)−4.51 ** ± 18.55263.4 ± 177.0240.1 ± 161.91.6295.42
Glycine (g)0.83 (0.76, 0.87)−10.47 ** ± 14.802.26 ± 1.421.93 ± 1.451.2992.37
Vitamin A (IU)0.86 (0.81, 0.90)28.00 ** ± 31.3213,294 ± 17,26516,373 ± 16,5642.7496.95
Vitamin C (mcg)0.88 (0.83, 0.91)2.41 ± 24.76149.4 ± 158.3149.1 ± 167.32.1693.13
Vitamin D (mcg)0.89 (0.84, 0.92)0.72 ± 14.533.90 ± 2.143.91 ± 2.201.2795.42
Vitamin E (mcg)0.82 (0.75, 0.87)−0.10 ± 24.316.68 ± 5.346.13 ± 4.912.1292.37
Zinc (mg)0.83 (0.77, 0.88)−12.49 ** ± 15.827.64 ± 4.556.19 ± 3.921.3892.37
Calcium (mg)0.51 (0.37, 0.62)1.46 ± 31.82548.4 ± 335.5570.8 ± 389.02.7893.13
Magnesium (mg)0.86 (0.81, 0.90)1.40 ± 15.69211.6 ± 162.7210.3 ± 159.61.3793.89
Iron (mg)0.86 (0.81, 0.90)3.11 ± 18.528.98 ± 6.028.99 ± 5.661.6294.66
Sodium (mg)0.75 (0.67, 0.82)−19.78 ** ± 18.842867 ± 18692002 ± 12661.6594.66
Note: NDSR: Nutrition Data System for Research; r: pairwise for intraclass correlation coefficient; CI: confidence interval; M: mean; SD: standard deviation; * p < 0.05; ** p < 0.001.
Table
Table 2. Differences between the internet-based application and Nutrition Data System for Research per domains of caloric ranges and various diets for essential macro-and micro-nutrients (n = 131).
Table 2. Differences between the internet-based application and Nutrition Data System for Research per domains of caloric ranges and various diets for essential macro-and micro-nutrients (n = 131).
Parameters (n)Calories, kcal
%Diff M ± SD		Carb, g
%Diff M ± SD		Protein, g
%Diff M ± SD		Fat, g
%Diff M ± SD		Folate, mcg
%Diff M ± SD		Cobalamin, mcg
%Diff M ± SD
Caloric range
<1000 (63)−4.26 **± 10.01−0.70 ** ± 12.55−6.27 ** ± 11.35−10.58 **± 13.7312.79 **± 18.86−13.43 **± 18.61
1000–2000 (50)−4.10 **± 11.675.15 **± 11.60−3.03 **± 9.04−14.34 **± 17.199.54 **± 10.94−12.27 **± 15.28
>2000 (18) −11.56 ± 25.71−9.98 ± 29.51−12.03 * ± 23.09−16.16 **± 23.74−5.35 ± 27.79−17.83 * ± 26.04
Diet types
Pure Liquid (8)−0.38 ± 7.322.88 ± 4.52−9.53 ± 17.28−0.66 ± 17.6520.05 ± 38.13−22.04 ** ± 23.39
Convenient Diet (30)−8.30 ** ± 12.814.18 ** ± 9.941.85 ± 10.41−23.65 ** ± 20.1620.47 ** ± 15.55−8.11 ** ± 12.91
Canned Food (10)−6.83 * ± 9.165.60 ** ± 4.153.84 ** ± 4.35−27.27 ** ± 17.5630.51 ** ± 15.86−8.96 ** ± 1.84
High School (10)0.55 ± 3.845.50 ** ± 2.640.42 ± 8.33−5.32 * ± 5.3519.10 ** ± 6.64−12.51 * ± 15.18
Fast Food (10)−18.61 ** ± 14.931.43 ± 16.781.30 ± 15.94−38.36 ** ± 18.5411.81 ** ± 16.93−2.87 ± 15.82
Ethnic Food (71)−3.22 ** ± 9.902.01 ± 13.96−7.40 ** ± 8.35−9.14 ** ± 8.537.10 ** ± 9.35−13.62 ** ± 16.05
Western Diet (38)−1.90 ± 9.662.86 *± 14.24−5.15 ** ± 7.57−6.53 **± 9.037.78 ** ± 9.60−12.82 **± 14.22
Mexican (10)3.40 ± 11.8313.04 * ± 17.65−5.01 ** ± 3.45−6.59 ± 11.8311.94 ** ± 9.62−14.35 ** ± 4.66
Italian (10)−1.31 ± 3.104.46 ** ± 1.98−2.95 ± 7.29−8.11 ** ± 3.765.02 ** ± 2.75−9.77 * ± 10.05
Mediterranean (9)−6.80 ± 8.68−3.11 ± 8.65−10.10 ± 9.10−8.94 ± 10.888.68 ** ± 6.48−27.93 ** ± 16.92
American (9)−3.53 ± 11.07−4.27 ± 16.67−2.81 ± 8.34−2.28 ± 7.415.32 ± 15.270.58 ± 6.08
Eastern Diet (33)−4.75 ** ± 10.101.03 ± 13.78−9.99 ± 8.57−12.15 ± 6.886.31 ** ± 9.14−14.55 ** ± 18.11
Japanese (10)−3.13 ** ± 2.201.92 * ± 2.86−4.57 ** ± 1.61−8.34 ** ± 3.769.44 ** ± 2.958.90 ** ± 3.65
Chinese (10)−6.85 ** ± 2.311.83 * ± 2.87−10.74 ** ± 1.57−16.71 ** ± 2.093.65 ** ± 0.83−20.62 ** ± 4.73
Korean (13)−4.38 ± 16.07−0.26 ± 22.16−13.57 ** ± 12.38−11.57 ** ± 9.095.94 * ± 14.20−27.91 ** ± 12.88
Smoothie (22)−9.13 ± 23.79−7.70 ± 27.16−9.86 * ± 21.40−14.12 ** ± 23.53−4.18 ± 25.22−17.89 ** ± 27.84
Note: M: mean; SD: standard deviation; n ≤ 30 used Signed-Rank test * p < 0.05; ** p < 0.001.
Table
Table 3. Significant factors contributing to the differences between the internet-based application against Nutrition Data System for Research on total calories.
Table 3. Significant factors contributing to the differences between the internet-based application against Nutrition Data System for Research on total calories.
ParametersLogistic Regression Original ModelGeneralized Regression
Elastic Net Model Validation
Estimate (95% CI)p (χ2)Estimate (95% CI)p (χ2)
(Intercept)3.1970 (1.8086, 4.5855) <0.00013.1367 (1.7804, 4.4933)<0.0001
<1000 caloric range−1.4626 (−2.7979, −0.1274)0.0318−1.4136 (−2.7006, −0.1265)0.0313
Fat % Difference−3.3411 (−4.6729, −2.009)<0.0001−3.2879 (−4.5876, −1.9883)<0.0001
Protein % Difference−1.3791 (−2.4530, −0.3052)0.0118−1.3600 (−2.4073, −0.3127)0.0109
MR0.18520.1852
AICc30.2930.29
AUC0.89200.8920
Note: MR: misclassification rate; AICc: Akaike’s information criterion with corrections; AUC: Area under the curve; CI: confidence interval.
Table
Table 4. Significant factors contributing to the differences between internet-based application against Nutrition Data System for Research on folate.
Table 4. Significant factors contributing to the differences between internet-based application against Nutrition Data System for Research on folate.
ParametersLogistic Regression Original ModelGeneralized Regression
Elastic Net Model Validation
Estimate (95% CI)p (χ2)Estimate (95% CI)p (χ2)
(Intercept)11.4831 (−139.76, 162.72)0.881710.3903 (9.0613, 11.7193)<0.0001
<1000 caloric range1.4607 (0.4860, 2.4354)0.00331.4604 (0.4854, 2.4354)0.0033
Carbohydrate, % difference−1.1903 (−2.9901, −0.93790.0001−1.9187 (−2.8978, −0.9395)0.0001
Italian diet−11.2367 (−162.48, 140.00)0.8842−10.1439 (−11.3112, −8.9767)<0.0001
MR0.29630.2963
AICc38.5238.52
AUC0.90460.9046
Note: MR: misclassification rate; AICc: Akaike’s information criterion with corrections; AUC: Area under the curve; CI: confidence interval.
Table
Table 5. Significant factors contributing to the differences between internet-based application against Nutrition Data System for Research on cobalamin.
Table 5. Significant factors contributing to the differences between internet-based application against Nutrition Data System for Research on cobalamin.
ParametersLogistic Regression Original ModelGeneralized Regression
Elastic Net Model Validation
Estimate (95% CI)p (χ2)Estimate (95% CI)p (χ2)
(Intercept)−23.3204 (−314.45, 267.81)0.8752−5.0243 (−6.7635, 3.2852)<0.0001
Protein, % difference−3.2596 (−4.4292, −2.0901)<0.0001−2.6890 (−3.6172, 1.7607)<0.0001
American diet12.3545 (−207.66, 232.37)0.91242.8874 (1.3699, 4.4048)0.0002
Japanese diet−23.3204 (−117.56, 203.75)0.87523.8314 (2.7588, 4.9040)<0.0001
MR0.25930.2593
AICc37.7337.42
AUC0.80830.8083
Note: MR: misclassification rate; AICc: Akaike’s information criterion with corrections; AUC: Area under the curve; CI: confidence interval.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Yang, Y.-L.; Yang, H.-L.; Kusuma, J.D.; Shiao, S.-Y.P.K. Validating Accuracy of an Internet-Based Application against USDA Computerized Nutrition Data System for Research on Essential Nutrients among Social-Ethnic Diets for the E-Health Era. Nutrients 2022, 14, 3168. https://doi.org/10.3390/nu14153168

AMA Style

Yang Y-L, Yang H-L, Kusuma JD, Shiao S-YPK. Validating Accuracy of an Internet-Based Application against USDA Computerized Nutrition Data System for Research on Essential Nutrients among Social-Ethnic Diets for the E-Health Era. Nutrients. 2022; 14(15):3168. https://doi.org/10.3390/nu14153168

Chicago/Turabian Style

Yang, Ya-Ling, Hsiao-Ling Yang, Joyce D. Kusuma, and Shyang-Yun Pamela Koong Shiao. 2022. "Validating Accuracy of an Internet-Based Application against USDA Computerized Nutrition Data System for Research on Essential Nutrients among Social-Ethnic Diets for the E-Health Era" Nutrients 14, no. 15: 3168. https://doi.org/10.3390/nu14153168

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop