Next Article in Journal
Use of Anti-Thrombotic Drugs and In-Hospital Mortality in Acute Aortic Dissection Patients
Previous Article in Journal
Network Analysis Reveals That Headache-Related, Psychological and Psycho–Physical Outcomes Represent Different Aspects in Women with Migraine
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Diagnostics
Volume 12
Issue 10
10.3390/diagnostics12102319
diagnostics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Intra- and Inter-Reader Variations in Lung Nodule Measurements: Influences of Nodule Size, Location, and Observers

by
Hong Chen
1,†,
Haozhe Huang
2,3,†,
Jianye Zhang
1,
Xuexue Wang
1,
Mengyang Han
1,
Chanjun Ding
1 and
Jinhong Wang
1,*
1
Department of Medical Imaging, Shanghai Mental Health Center, Shanghai Jiao Tong University School of Medicine, Shanghai 200030, China
2
Department of Interventional Radiology, Fudan University Shanghai Cancer Center, 270 Dongan Road, Xuhui District, Shanghai 200032, China
3
Department of Oncology, Shanghai Medical College, Fudan University, 130 Dongan Road, Xuhui District, Shanghai 200032, China
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Diagnostics 2022, 12(10), 2319; https://doi.org/10.3390/diagnostics12102319
Submission received: 29 August 2022 / Revised: 21 September 2022 / Accepted: 21 September 2022 / Published: 26 September 2022
(This article belongs to the Section Medical Imaging and Theranostics)
Browse Figures
Versions Notes

Abstract

:
(1) Background: Accurate measurement of lung-nodule size is necessary, but whether a three-dimensional volume measurement is better or more reliable than the one-dimensional method is still unclear. This study aimed to investigate the intra- and inter-reader variations according to nodule type, size, three-dimensional volume measurements, and one-dimensional linear measurements. (2) Methods: This retrospective study included computed tomography (CT) examinations of lung nodules and volume measurements performed from October to December 2016. Two radiologists independently performed all measurements. Intra-class correlation coefficients (ICC) and Bland-Altman plots were used for analysis. (3) Results: The overall variability in the calculated volume was larger than when using the semiautomatic volume measurement. Nodules <6 mm tended to have larger variability than nodules ≥6 mm in both one-dimensional and calculated volume measurements. The isolated type showed smaller variability in both intra- and inter-reader comparisons. The juxta-vascular type showed the largest variability in both one-dimensional and calculated volume measurements. The variability was decreased when using the 3D volume semiautomated software. (4) Conclusions: The present study suggests that 3D semiautomatic volume measurements showed lower variability than the calculated volume measurement. Nodule size and location influence measurement variability. The intra- and inter-reader variabilities in nodule volume measurement were considerable.

1. Introduction

Lung cancer is the most harmful malignant tumor in public health, with a high incidence and serious disease burden [1]. According to the latest report of the International Cancer Research Institute of the World Health Organization in 2022, the mortality of lung cancer ranks first among malignant tumors and at 2.5 times higher than colorectal cancers, the second leading cause of cancer death [1,2]. The prognosis of lung cancer is closely related to the clinical stage. The 5 year survival rate of patients with early lung cancer can reach 59% after surgery, while it is only 6% for patients with advanced lung cancer after treatment [2]. Therefore, early detection and early diagnosis of lung cancer is the key to the survival and prognosis of patients [3]. At present, low-dose CT, as the primary means of lung cancer screening, has been widely recommended by authoritative clinical guidelines [4,5,6,7,8]. The grading management of pulmonary nodules after screening mainly depends on the size of pulmonary nodules and the changes in size during the follow-up period [8,9,10,11]. Therefore, accurately measuring the diameter and volume of pulmonary nodules is crucial to the development of management and follow-up strategies for pulmonary nodules.
Scanning and reconstruction parameters, nodule shape, lung volume, lung parenchyma on CT images, nodule size, location, readers’ observation, measurement methods, and software can affect nodule size [12,13,14,15], leading to erroneous estimations of the growth and the inaccurate prediction of the nodules. In 1979, the World Health Organization (WHO) first introduced a criterion based on the change in the size of the lesion by determining the largest diameter and its perpendicular length (bidimensional measurement) for each measurable lesion [16]. Later, a study showed that one-dimensional measurements of tumor lesions might better identify the area of tumor killed by an anti-neoplastic agent rather than the bidimensional measurement [17]. These criteria use a one-dimensional measurement instead of a bi-dimensional measurement to determine treatment response [18]. The one-dimensional measurement might fail to detect irregular growth since tumor growth is a three-dimensional (3D) phenomenon, and the growth pattern is not necessarily spherical [19]. The Fleischner Association’s 2017 guidelines recommended that the diameter of pulmonary nodules be expressed as the average of their maximum length diameter and vertical diameter [8]. In 2019, the lung imaging reporting and data system (Lung CT Screening Reporting and Data System, Lung-RADS) 1.1 guidance by the American College of Radiology added the volume of pulmonary nodules to the grading standard and stipulated that the measurement results of all pulmonary nodules’ diameters and volumes should be in mm [20].
Furthermore, with the development of computer-aided diagnosis techniques, 3D measurements are now routinely used in clinical applications. Nevertheless, it remains undefined whether the 3D volumetric measurement of pulmonary nodules is superior to and more reliable than the one-dimensional method when assessing lung nodules. Therefore, this study investigated the intra- and inter-reader variations according to nodule type, size, 3D volumetric measurements, and one-dimensional linear measurements to find an accurate method to measure pulmonary nodules size. The results could help provide a basis for evaluating and managing pulmonary nodules. Then, the structure of our research is presented from the following aspects: Section 2 explains the methodology, Section 3 clarifies the results, Section 4 makes a discussion, and the last section draws conclusions.

2. Material and Methods

2.1. Study Design and Nodule Selection

This retrospective study included CT examinations of lung nodules. The subjects who underwent CT between October and December 2016 in Shanghai Mental Health Center XX were selected. All eligible lung nodule image sets during this period were examined. Calcified or cavitary nodules and image sets with poor image quality were excluded. The study was approved by the ethics committee.

2.2. CT Scanning

All chest CT examinations were performed using a spiral multi-detector CT scanner (GE Healthcare, Waukesha, WI, USA) without contrast. The low-dose screening protocol was used: 120 kVp, 30 effective mAs, and single-breath holding. The scanning range was from the chest entrance to the bottom of the lung, including the entirety of the lungs. The images were reconstructed with a standard lung kernel at a section thickness of 2 mm, without an intersection gap, field of view 350 mm × 350 mm, and matrix 512 × 512, according to the routine screening protocol.

2.3. Linear and Volumetric Measurements of the Nodule

Two experienced chest CT radiologists (readers A and B, both with >10 years of experience) were instructed to first identify and measure the lung nodules on the pre-selected images independently. Measurements were performed on the screening CT examinations using the largest diameter of the nodule in the linear unidimensional display. Multiple measurements were repeatedly performed to determine the largest diameter. The largest diameters were used to calculate the volume of the lung nodules using the formula: V = 4/3πr3. Then, 3D semiautomatic volume measurements were performed using the 3D display. Reader A repeated both linear unidimensional and volumetric measurements at a 1-week interval while being blinded to the results from the previous measurements. Reader B only made the measurements once. A database was constructed from the linear and volumetric measurements of each nodule.
For 3D semi-automatic volume measurements, the Lung CARE volumetry software (GE Healthcare, Waukesha, WI, USA) was used. The center of the nodule was selected in the axial plane. When the nodule was marked, the initial 3D template originating from the center point was produced in color. The 3D volume of the nodule was automatically obtained. If a nodule touched vessels or the pleura, the first measurement result was not satisfying because the volume of a nodule may not cover the whole nodule or extend to the adjacent vessels or pleura. The readers manually adjusted the included 3D volume by volume decrease or increase according to the coverage of the nodule in the one-dimensional axial plane.

2.4. Definitions of Nodules

Nodules were categorized into three types based on their location within the lungs: isolated, juxta-vascular, and juxta-pleural. Isolated nodules were completely surrounded by aerated lung parenchyma. Juxta-pleural nodules were located within 2 mm of the pleura. Juxta-vascular nodules were sitting on or adjacent to blood vessels (Figure 1). Nodule size was defined as the largest diameter in the CT axial plane. If a nodule was found on several consecutive images, the image depicting the largest diameter was selected. According to the 2017 Fleischner Association guidelines, lung nodules were divided into three groups by size: (1) <6 mm; (2) ≥6 and <8 mm; and (3) ≥8 mm [8]. The guidelines set the evaluation threshold of the nodules as 6 and 8 mm and nodules with a diameter of ≥6 mm should be actively dealt with.

2.5. Statistical Analysis

The continuous data were presented as means ± standard deviation or as median (range) according to their distribution, determined using the Kolmogorov-Smirnov test. Categorical data were presented as numbers (percentages). The intra-reader difference was determined using the original and repeated measurements by reader A. The inter-reader difference was derived from the first measurement of reader A, and the measurement of reader B. Measurements were analyzed based on different nodule types and sizes (Figure 1).
The intraclass correlation coefficient (ICC) was used to evaluate the reliability of the measurements. Its value was between 0 and 1. A value of 0 meant untrusted and 1 meant completely trusted. It was generally believed that a reliability coefficient lower than 0.4 indicated poor reliability, while a reliability coefficient greater than 0.75 indicated good reliability. Higher ICC values were needed for quantitative data. A scatter plot was used to explore the distribution law of the two measurements methods of the lung nodules. By observing the distribution of data points on the scatter plot, we could infer the correlation between variables. The Bland-Altman plot was used To further assess the intra- and inter-reader differences [21]. MedCalc software (MedCalc Software bvba, Ostend, Flanders, Belgium) was used for statistical analysis. Two-sided p-values <0.05 were considered statistically significant.

3. Results

3.1. Characteristics of the Patients and Nodules

The CT examinations of 56 patients performed from October to December 2016 were included. There were 29 (51.8%) men and 27 (48.2%) women, and the mean age was 61.7 (21–85) years. There were 83 lung nodules among the 56 patients. The longest diameter range of the nodules was 3.3–18.2 mm. There were 41 (49.4%) nodules <6 mm, 29 (34.9%) nodules of ≥6 and <8 mm, and 13 (15.7%) nodules ≥8 mm. The nodules were categorized as isolated (n = 31), juxta-vascular (n = 19), and juxta-pleural nodules (n = 33). Table 1 shows the descriptive statistics of the measurements obtained by readers A and B.

3.2. Intraclass Correlation Coefficients

To explore the reliability between the calculated volume using the longest diameter and 3D semiautomatic volume, Table 2 presents the ICC for the calculated volume using the longest diameter and the 3D semiautomatic volume between different nodule types, different sizes, and different readers. The measurement of all nodules showed good reliability, but there were differences among different nodule types and different sizes. For different nodule sizes, the nodules ≥8 mm showed relatively larger ICC (>0.9) than the nodules with other sizes (<8 mm). For different nodule types, the isolated nodule type showed a relatively greater ICC (>0.9) than the other types.
Figure 2 shows the distribution law of the two measurement methods of lung nodules. By observing the distribution of data points on the scatter plot, we inferred the correlation between the overall data. The dotted line shows the scatter plot of 56 patients’ calculated volume using the longest diameter. The solid line shows the scatter plot of 56 patients’ 3D semiautomatic volume measurement. The dispersion of values in the solid line is smaller than that in the dotted line, indicating a small degree of variability in the volume of 3D semi-automatic measurement of 56 patients. However, the overall trend of the two measurement methods is similar. The volume obtained by the long diameter measurement is larger than that obtained by the 3D semi-automatic measurement, which may be due to the use of the longest diameter to calculate the volume.
Table 3 presents the ICCs of the intra- and inter-reader measurements based on overall nodules, different sizes, and different types to explore the reliability between different readers and repeated readings. The overall data showed good reliability, but there were differences between different measurement methods. Furthermore, the intra-reader measurements showed greater ICCs than inter-reader measurements, especially in 3D semiautomatic volume measurements between A1 and 2. For different nodule sizes, the nodules ≥8 mm showed relatively larger ICCs than the smaller nodules (<8 mm) in both intra- and inter-reader measurement comparisons. The ICC for the inter-reader calculated volume measurements tended to increase along with the increasing nodule size. For different nodule types, there were no obvious differences in the intra-reader comparison, but the inter-reader 3D measurement of the isolated nodule type showed a relatively greater ICC (1.000) than the other types.

3.3. Bland-Altman Plots for All Measurements

Table 4 presents the Bland-Altman plots and correlation results deriving from the overall nodule measurements. In the intra- and inter-reader comparisons, the limits of agreement of the 3D semiautomatic volume measurements were smaller than for the calculated volume measurements. Hence, the inter-reader comparison showed larger limits of agreement than that of the intra-reader comparison.

3.4. Bland-Altman Plots Deriving from Different Nodule Size

Table 5 presents the results of the mean differences and limits of agreement for the different nodule types. The limits of agreement of the diameter measurement in nodules <6 mm were larger than in the larger nodules (≥6 mm) in intra- and inter-reader comparisons. Thus, the calculated volume, which was calculated using the longest diameter, followed a similar tendency as the diameter measurements, but it had larger limits of agreement. In the intra-reader comparison, the smallest limit of agreement of 3D semiautomatic volume measurement was observed in nodules <6 mm, but in the inter-reader comparison, the smallest limit of agreement was observed in nodules ≥8 mm.

3.5. Bland-Altman Plots Deriving from Different Nodule Types

Table 6 presents the results of the mean differences and limits of agreement deriving from different nodule types. The limits of agreement for the isolated nodule type were smaller than for the other nodule types in intra- and inter-reader comparisons. On the other hand, the juxta-vascular nodule type showed the largest limits of agreement except in the intra-reader 3D volume measurements.

4. Discussion

Accurate measurement of lung nodule size is necessary to determine its benign or malignant nature or determine its response to treatment, especially in nodule follow-up management [9,10,11,22,23]. But it remains undefined whether 3D volumetric measurement is superior to or more reliable than the one-dimensional method. Therefore, this study aimed to investigate the contribution of nodule type, size, and different readers to the differences between 3D volumetric and one-dimensional linear measurements of lung nodules. The results suggest that the semiautomatic volume measurements showed lower variability than the calculated volume measurements. However, the measurement of the Juxta-vascular nodules type should be more careful. The type nodules are vulnerable to the influence by vascular shadows, which may lead to measurement deviation (Table 6). Nodule size and location influence measurement variability. The intra-reader and inter-reader variabilities in nodule volume measurements were considerable. In the follow-up management of nodules, it is more likely to be followed up by the same observer to increase the reliability of measurement. Previous studies indicated that the one-dimensional measurement type could replace the more complex bi-dimensional measurement [17,24]. Nevertheless, each measurement has some difficulties in estimating the size of irregular or confluent lesions and discrepancies arise in scan planes and patient positioning [10,25]. Advanced CT technology and new 3D semiautomatic volumetric measurement software are thought to be better methods to quantitatively assess the total tumor volume and associated masses [9,11,14]. The present study was designed thoroughly compare one-dimensional (linear) and 3D semiautomatic volumetric measurements for small lung nodules with large sample size.
The results showed that the overall variation in 3D volumetric measurement was smaller than that of the calculated volume using linear measurement (Figure 2 and Table 4). Each reading and the overall data showed good reliability, which means that both the calculated volume measurement using the longest diameter and the 3D semiautomatic volume measurement is reliable. But the 3D semiautomatic volume measurement showed a smaller degree of variability. The main influencing factors are the non-isolated nodules and smaller-size nodules (<8 mm) (Table 3 and Table 5). In this study, 63% (52 of 83) of the lung nodules were juxta-structure (juxta-vascular or juxta-pleural) nodules, which might be considered irregular to some extent. Linear measurements can hardly estimate the volume of irregular lesions because the nodules vary according to the measurement plane. The nodules are not perfectly spherical, and the adjacent structure deforms or truncates the lesion. Furthermore, a small variation in diameter measurement might cause large volumetric variations when using the formula V = 4/3πr3 since the measurement error is exponential and multiplied by 4/3π. On the other hand, the volume obtained using semiautomatic 3D measurements is calculated by counting the number of voxels directly in a nodule. Therefore, those are probably the reasons for the important difference in volume measurement variability between the two methods of nodule volume measurements. A previous study showed that 3D measurements allow an objective evaluation, reducing inter-observer variability in evaluating chemotherapy response of lung tumors [26]. Therefore, the 3D measurement could be superior to the traditional linear method.
The analysis of the nodule size groups showed that smaller nodules had the largest variations in calculated volume measurement. In the intra-reader comparison of 3D volume measurement, the variation of each nodule size was smaller than in the calculated volume measurement, but the variation tended to increase with decreasing nodule size. On the other hand, it was not observed in inter-reader 3D volume measurement. Therefore, the 3D software could help decrease the variations in volume measurements, especially when the measurements are performed by the same reader. For different readers, manual adjustments might lead to large variations, especially when the nodules are small. Thus, using a 3D computed semiautomatic software to measure lung nodules by different readers might reduce its reliability. A previous study reported that there was relatively good agreement among nine consecutive measurements made by three independent readers [27] using the GE lung analysis software. In the present study, there was a larger number of juxta-structure nodules (63%, 52 of 83), which may have highly irregular shapes.
The analysis of nodule types suggested that the isolated nodule had smaller variations in size/volume than the other types. The juxta-vascular type showed the largest variations in linear measurement and calculated volume measurement. Nevertheless, the measurement variation of the juxta-vascular type could be reduced using 3D volume measurements performed by the same reader. Thus, adjacent structures might cause inaccurate manual measurements, but well-designed 3D semi-automatic software can help identify the boundaries between nodules and adjacent structures effectively. A previous phantom study on the effects of blood vessels on nodule volume measurement reported a partially similar finding [28]; it indicated that the presence of adjacent vessels might produce the linear measurement errors. This difference probably results from the presence of the pulmonary structures with attenuation values like pulmonary nodules, and that may extend or reduce the segmentation range from the background. Although recent studies examined isolated, juxta-vascular, and juxta-pleural nodules, the variability in volume measurements has not been thoroughly investigated for the juxta-structure nodules [9,15,29,30].
Although inter- and intra-reader variability in nodule volume measurement has been extensively studied [29,30,31,32], the intra-reader variability varied among studies. Erasmus [32] reported that the inter-reader variability of diameter measurements was greater than that of intra-reader measurement. Revel et al. [33] reported poor inter-reader reliability in diameter measurements. In lung cancer, Wang et al. [34] reported that the intra-observer agreement was satisfying, while the inter-observer agreement was limited. The results reported here delineated a great variability between intra-reader and inter-reader comparisons. The inter-reader variation was always larger than that of the intra-reader comparison. The 3D volume measurement showed the smallest variation compared with other methods in the intra-reader comparison, while this was not that prominent in the inter-reader comparison. The reason might be that the 3D software used here was semiautomatic, and the variations in the determination of the nodule center and the manual adjustments of the nodule boundaries by different readers were significant.
The present study had limitations. The true values of the diameter and volume of the pulmonary nodules were unknown because the nodules were not removed in many cases. Measurement precision is the most important aspect of pulmonary nodule volumetric analysis for assessing lesion growth with low-dose CT. Another limitation is that the 3D volumetric software seldom enables the segmentation of juxta-pleural or juxta-vascular nodules, although a visual control of the semiautomatic segmentation had to be used by overlaying the images with color-coded segmentation results.

5. Conclusions

Accurate measurement of the diameter and volume of pulmonary nodules is crucial for the diagnosis and follow-up strategy of pulmonary nodules. The present study indicates that 3D semiautomatic volume software shows a smaller variation than the calculated volume method using a linear measurement. Nodule size and location influence measurement variability. The variability among different readers in nodule volume measurement can be considerable. The results of the present study suggest that a 3D volume software is a better choice than the calculated volume, when possible. Baseline and follow-up measurements should be performed by the same reader whenever possible. For small and juxta-structure nodules, performing multiple measurements and calculating a mean value as the result could be a good method.

Author Contributions

Conceptualization, J.W. and H.C.; methodology, H.H. and J.W.; software, H.C.; validation, H.C. and J.Z.; formal analysis, H.H., X.W.; investigation, M.H.; resources, J.W.; data curation, C.D.; writing—original draft preparation, H.C.; writing—review and editing, H.C., H.H. and J.W.; visualization, H.C.; supervision, J.W.; project administration, J.W. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by grants from the hospital project (2018-YJ-13).

Institutional Review Board Statement

This study is a retrospective analysis of image measurement. According to the regulations of this ethics committee, this study does not require ethical application and informed consent of patients.

Informed Consent Statement

Patient consent was waived due to this single-center retrospective study.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank the study participants for their contribution.

Conflicts of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

  1. Siegel, R.L.; Miller, K.D.; Fuchs, H.E.; Jemal, A. Cancer statistics, 2022. CA A Cancer J. Clin. 2022, 72, 7–33. [Google Scholar] [CrossRef] [PubMed]
  2. Global Burden of Disease Cancer Collaboration; Fitzmaurice, C.; Abate, D.; Abbasi, N.; Abbastabar, H.; Abd-Allah, F.; Abdel-Rahman, O.; Abdelalim, A.; Abdoli, A.; Abdollahpour, I.; et al. Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived with Disability, and Disability-Adjusted LifeYears for 29 Cancer Groups, 1990 to 2017: A Systematic Analysis for the Global Burden of Disease Study. JAMA Oncol. 2019, 5, 1749–1768. [Google Scholar]
  3. Gasparri, R.; Guaglio, A.; Spaggiari, L. Early Diagnosis of Lung Cancer: The Urgent Need of a Clinical Test. J. Clin. Med. 2022, 11, 4398. [Google Scholar] [CrossRef]
  4. Hoffman, R.M.; Atallah, R.P.; Struble, R.D.; Badgett, R.G. Lung Cancer Screening with Low-Dose CT: A Meta-Analysis. J. Gen. Intern. Med. 2020, 35, 3015–3025. [Google Scholar] [CrossRef] [PubMed]
  5. Oudkerk, M.; Liu, S.Y.; Heuvelmans, M.A.; Walter, J.E.; Field, J.K. Lung cancer LDCT screening and mortality reduction—Evidence, pitfalls and future perspectives. Nat. Rev. Clin. Oncol. 2021, 18, 135–151. [Google Scholar] [CrossRef]
  6. Becker, N.; Motsch, E.; Trotter, A.; Heussel, C.P.; Delorme, S. Lung cancer mortality reduction by LDCT screening—Results from the randomized German LUSI trial. Int. J. Cancer 2019, 146, 1503–1513. [Google Scholar] [CrossRef]
  7. Wood, D.E.; Kazerooni, E.A.; Aberle, D.; Berman, A.; Brown, L.M.; Eapen, G.A.; Ettinger, D.S.; Ferguson, J.S.; Hou, L.; Kadaria, D.; et al. NCCN Guidelines® Insights: Lung Cancer Screening, Version 1.2022. J. Natl. Compr. Cancer Netw. 2022, 20, 754–764. [Google Scholar] [CrossRef]
  8. Macmahon, H.; Naidich, D.P.; Goo, J.M.; Lee, K.S.; Leung, A.N.C.; Mayo, J.R.; Mehta, A.C.; Ohno, Y.; Powell, C.A.; Prokop, M. Guidelines for Management of Incidental Pulmonary Nodules Detected on CT Images: From the Fleischner Society 2017. Radiology 2017, 284, 228–243. [Google Scholar] [CrossRef]
  9. Han, D.; Heuvelmans, M.A.; Oudkerk, M. Volume versus diameter assessment of small pulmonary nodules in CT lung cancer screening. Transl. Lung Cancer Res. 2017, 6, 52–61. [Google Scholar] [CrossRef]
  10. Devaraj, A.; van Ginneken, B.; Nair, A.; Baldwin, D. Use of Volumetry for Lung Nodule Management: Theory and Practice. Radiology 2017, 284, 630–644. [Google Scholar] [CrossRef] [PubMed]
  11. Bankier, A.A.; MacMahon, H.; Goo, J.M.; Rubin, G.D.; Schaefer-Prokop, C.M.; Naidich, D.P. Recommendations for Measuring Pulmonary Nodules at CT: A Statement from the Fleischner Society. Radiology 2017, 285, 584–600. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Gierada, D.S.; Rydzak, C.E.; Zei, M.; Rhea, L. Improved Interobserver Agreement on Lung-RADS Classification of Solid Nodules Using Semiautomated CT Volumetry. Radiology 2020, 297, 200302. [Google Scholar] [CrossRef]
  13. Sui, X.; Meinel, F.G.; Song, W.; Xu, X.; Wang, Z.; Wang, Y.; Jin, Z.; Chen, J.; Vliegenthart, R.; Schoepf, U.J. Detection and size measurements of pulmonary nodules in ultra-low-dose CT with iterative reconstruction compared to low dose CT. Eur. J. Radiol. 2016, 85, 564–570. [Google Scholar] [CrossRef] [PubMed]
  14. Ying, R.Z.; Ooijen, P.; Dorrius, M.; Heuvelmans, M.; Oudkerk, M. Comparison of three software systems for semi-automatic volumetry of pulmonary nodules on baseline and follow-up CT examinations. Acta Radiol. 2014, 55, 691–698. [Google Scholar]
  15. Willemink, M.J.; Leiner, T.; Budde, R.P.J.; de Kort, F.P.L.; Vliegenthart, R.; van Ooijen, P.M.A.; Oudkerk, M.; de Jong, P.A. Systematic error in lung nodule volumetry: Effect of iterative reconstruction versus filtered back projection at different CT parameters. Ajr Am. J. Roentgenol. 2012, 199, 1241–1246. [Google Scholar] [CrossRef]
  16. WHO. WHO Handbook for Reporting Results of Cancer Treatment; World Health Organization Offset Publication: Geneva, Switzerland, 1979; p. 48. [Google Scholar]
  17. James, K.; Eisenhauer, E.; Christian, M.; Terenziani, M.; Vena, D.; Muldal, A.; Therasse, P. Measuring response in solid tumors: Unidimensional versus bidimensional measurement. J. Natl. Cancer Inst. 1999, 91, 523–528. [Google Scholar] [CrossRef] [PubMed]
  18. Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur. J. Cancer 2009, 45, 228–247. [Google Scholar] [CrossRef] [PubMed]
  19. Gonzalez, M.M.; Joa, J.A.; Cabrales, L.E.; Pupo, A.E.; Schneider, B.; Kondakci, S.; Ciria, H.M.; Reyes, J.B.; Jarque, M.V.; Mateus, M.A.; et al. Is cancer a pure growth curve or does it follow a kinetics of dynamical structural transformation? BMC Cancer 2017, 17, 174. [Google Scholar] [CrossRef]
  20. Chelala, L.; Hossain, R.; Kazerooni, E.A.; Christensen, J.D.; Dyer, D.S.; White, C.S. Lung-RADS Version 1.1: Challenges and a Look Ahead, From the AJR Special Series on Radiology Reporting and Data Systems. AJR Am. J. Roentgenol. 2021, 216, 1411–1422. [Google Scholar] [CrossRef]
  21. Bland, J.M.; Altman, D.G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999, 8, 135–160. [Google Scholar] [CrossRef]
  22. Gould, M.K.; Fletcher, J.; Iannettoni, M.; Lynch, W.R.; Midthun, D.E.; Naidich, D.P.; Ost, D.E. Evaluation of Patients with Pulmonary Nodules: When Is It Lung Cancer? ACCP Evidence-Based Clinical Practice Guidelines (2nd edition). Chest 2007, 132, IL. [Google Scholar] [CrossRef] [PubMed]
  23. Gould, M.K.; Donington, J.; Lynch, W.R.; Mazzone, P.J.; Midthun, D.E.; Naidich, D.P.; Wiener, R.S. Evaluation of Individuals With Pulmonary Nodules: When Is It Lung Cancer? Diagnosis and Management of Lung Cancer, 3rd ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2013, 143, e93S–e120S. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Prasad, S.R.; Jhaveri, K.S.; Sanjay, S.; Hahn, P.F.; Halpern, E.F.; Sumner, J.E. CT tumor measurement for therapeutic response assessment: Comparison of unidimensional, bidimensional, and volumetric techniques initial observations. Radiology 2002, 225, 416. [Google Scholar] [CrossRef] [PubMed]
  25. Cortes, J.; Rodriguez, J.; Diaz-Gonzalez, J.A.; Garzon, C.; Gurpide, A.; Arbea, L.; Gil-Bazo, I.; Navarro, V.; Cambeiro, M.; Nicolas, A.I.; et al. Comparison of unidimensional and bidimensional measurements in metastatic non-small cell lung cancer. Br. J. Cancer 2002, 87, 158–160. [Google Scholar] [CrossRef] [PubMed]
  26. Fraioli, F.; Bertoletti, L.; Napoli, A.; Calabrese, F.A.; Masciangelo, R.; Cortesi, E.; Catalano, C.; Passariello, R. Volumetric evaluation of therapy response in patients with lung metastases. Preliminary results with a computer system (CAD) and comparison with unidimensional measurements. Radiol. Med. 2006, 111, 365–375. [Google Scholar] [CrossRef]
  27. Revel, M.P.; Lefort, C.; Bissery, A.; Bienvenu, M.; Aycard, L.; Chatellier, G.; Frija, G. Pulmonary nodules: Preliminary experience with three-dimensional evaluation. Radiology 2004, 231, 459–466. [Google Scholar] [CrossRef]
  28. Ko, J.P.; Marcus, R.; Bomsztyk, E.; Babb, J.S.; Stefanescu, C.; Kaur, M.; Naidich, D.P.; Rusinek, H. Effect of blood vessels on measurement of nodule volume in a chest phantom. Radiology 2006, 239, 79–85. [Google Scholar] [CrossRef]
  29. Goodman, L.R.; Gulsun, M.; Washington, L.; Nagy, P.G.; Piacsek, K.L. Inherent variability of CT lung nodule measurements in vivo using semiautomated volumetric measurements. AJR Am. J. Roentgenol. 2006, 186, 989–994. [Google Scholar] [CrossRef]
  30. Volterrani, L.; Mazzei, M.A.; Scialpi, M.; Carcano, M.; Carbone, S.F.; Ricci, V.; Guazzi, G.; Lupattelli, L. Three-dimensional analysis of pulmonary nodules by MSCT with Advanced Lung Analysis (ALA1) software. La Radiol. Med. 2006, 111, 343–354. [Google Scholar] [CrossRef]
  31. Bolte, H.; Jahnke, T.; Schafer, F.K.; Wenke, R.; Hoffmann, B.; Freitag-Wolf, S.; Dicken, V.; Kuhnigk, J.M.; Lohmann, J.; Voss, S.; et al. Interobserver-variability of lung nodule volumetry considering different segmentation algorithms and observer training levels. Eur. J. Radiol. 2007, 64, 285–295. [Google Scholar] [CrossRef]
  32. Bogot, N.R.; Kazerooni, E.A.; Kelly, A.M.; Quint, L.E.; Desjardins, B.; Nan, B. Interobserver and intraobserver variability in the assessment of pulmonary nodule size on CT using film and computer display methods. Acad. Radiol. 2005, 12, 948–956. [Google Scholar] [CrossRef] [PubMed]
  33. Revel, M.P.; Bissery, A.; Bienvenu, M.; Aycard, L.; Lefort, C.; Frija, G. Are two-dimensional CT measurements of small noncalcified pulmonary nodules reliable? Radiology 2004, 231, 453–458. [Google Scholar] [CrossRef] [PubMed]
  34. Wang, Q.; Zhang, Z.; Shan, F.; Shi, Y.; Xing, W.; Shi, L.; Zhang, X. Intra-observer and inter-observer agreements for the measurement of dual-input whole tumor computed tomography perfusion in patients with lung cancer: Influences of the size and inner-air density of tumors. Thorac. Cancer 2017, 8, 427–435. [Google Scholar] [CrossRef] [PubMed]
Diagnostics 12 02319 g001 550
Figure 1. Lung window and 3D display of three types of nodules using the Lung CARE software. (A,D) are the juxta-vascular type. (B,E) are the juxta-pleural type. (C,F) are the isolated nodule type.
Figure 1. Lung window and 3D display of three types of nodules using the Lung CARE software. (A,D) are the juxta-vascular type. (B,E) are the juxta-pleural type. (C,F) are the isolated nodule type.
Diagnostics 12 02319 g001
Diagnostics 12 02319 g002 550
Figure 2. Scatter plots of the two volume measurements. The solid line represents 3D volume, which shows the scatter plot of 56 patients’ 3D semiautomatic volume measurement. The dotted line represents Volume, which shows the scatter plot of 56 patients’ calculated volume using the longest diameter.
Figure 2. Scatter plots of the two volume measurements. The solid line represents 3D volume, which shows the scatter plot of 56 patients’ 3D semiautomatic volume measurement. The dotted line represents Volume, which shows the scatter plot of 56 patients’ calculated volume using the longest diameter.
Diagnostics 12 02319 g002
Table
Table 1. Descriptive statistics of the data obtained by the two readers.
Table 1. Descriptive statistics of the data obtained by the two readers.
MeasurementsReader AReader B
A1A2
Longest diameter (mm)6.4 ± 2.5 (3.4–18.3)6.4 ± 2.6 (3.1–18.1)6.3 ± 2.5 (3.0–18.0)
The calculated volume of the nodule (mm3)222.6 ± 457.5 (20.6–3208.9)224.7 ± 482.4 (15.6–3104.8)212.9 ± 433.8 (14.1–3053.6)
3D semiautomatic volume (mm3)86.8 ± 278.4 (4.89–2433.9)86.7 ± 278.5 (4.9–2436.3)83.5 ± 252.4 (4.4–2179.6)
Data are shown as means ± standard deviations. The numbers in parentheses are the ranges. A1 is the first measurement by reader A. A2 is the repeated measurement one week later by reader A.
Table
Table 2. ICCs of overall measurement by size, type, and reader.
Table 2. ICCs of overall measurement by size, type, and reader.
Measurement ComparisonsOverallBy Size(mm)By Type
<6≥6 and <8≥8IsolatedJuxta-VascularJuxta-Pleural
A1 *
c_volume-volume **0.9160.7730.5520.9890.9680.6390.794
A2
c_volume-volume0.9000.7390.5420.9660.9810.6350.721
B1
c_volume-volume0.9010.7420.5400.9840.9790.6320.747
Overall
c_volume-volume0.9050.7480.5460.9770.9760.6420.748
* A1 is the first measurement by reader A. A2 is the repeated measurement one week later by reader A. B1 is the first measurement by reader B. Overall is a summary of the relevant measurements. ** c_volume-volume means the intraclass correlation coefficient (ICC) of the calculated volume using the longest diameter and the 3D semiautomatic volume.
Table
Table 3. Intra-reader and inter-reader measurements.
Table 3. Intra-reader and inter-reader measurements.
Measurement ComparisonsOverallBy Size (mm)By Type
<6 mm≥6 and<8 mm≥8 mmIsolatedJuxta-VascularJuxta-Pleura
Intra-reader
A1diameter-A2 diameter *0.9970.9800.9770.9940.9980.9950.997
A1 volume-A2 volume *1.0000.9990.9991.0001.0000.9991.000
A1_c_volume-A2_c_volume *0.9920.9830.9840.9880.9940.9990.991
Inter-reader
A1 diameter-B1 diameter0.9950.9410.9420.9960.9950.9890.996
A1volume-B1 volume0.9940.9870.9870.9941.0000.9860.993
A1_c_volume-B1_c_volume0.9980.9440.9580.9970.9860.9920.999
* A1 diameter-A2 diameter means the intraclass correlation coefficient (ICC) of the longest diameters of the lung nodules between the first (A1) and second (A2) measurements by reader A. A1 volume-A2 volume means the ICC for the 3D semiautomatic volume measurements between the first (A1) and second (A2) measurements by reader A. A1_c_volume-A2_c_volume means the ICC for the calculated volume using the longest diameter between the first (A1) and second (A2) measurements by reader A. B1 is the first measurement by reader B.
Table
Table 4. Mean differences and limits of agreement of overall measurement.
Table 4. Mean differences and limits of agreement of overall measurement.
Measurement ComparisonsMean Difference % (95% CI)95% Limits of Agreement
Intra-reader
A1 diameter -A2 diameter *−0.5 (−1.13, 0.13)−6.2, 5.2
A1 volume-A2 volume *0.2 (−0.25, 0.73)−4.1, 4.6
A1_c_volume-A2_c_volume *−1.5 (−3.36, 0.39)−18.3, 15.3
Inter-reader
A1 diameter -B1 diameter−1.5 (−2.48, −0.44)−10.6, 7.7
A1 volume-B1 volume−1 (−3.19, 1.25)−20.9, 18.9
A1_c_volume-B1_c_volume−4.3 (−7.35, −1.33)−31.4, 22.7
Numbers in brackets are the 95% confidence intervals. * A1 diameter-A2 diameter means the intraclass correlation coefficient (ICC) of the longest diameters of the lung nodules between the first (A1) and second (A2) measurements by reader A. A1 volume-A2 volume means the ICC for the 3D semiautomatic volume measurements between the first (A1) and second (A2) measurements by reader A. B1 is the first measurement by reader B.
Table
Table 5. Mean differences and limits of agreement of different nodule sizes.
Table 5. Mean differences and limits of agreement of different nodule sizes.
Measurement Comparisons<6 mm≥6 and <8 mm≥8 mm
Mean Difference
% (95% CI)		95% Limits of AgreementMean Difference
% (95% CI)		95% Limits of AgreementMean Difference
% (95% CI)		95% Limits of Agreement
Intra-reader
A1 diameter-A2 diameter −1 (−2.03, 0.13)−7.7, 5.80.2 (−0.57, 0.95)−3.7, 4.1−0.6 (−2.19, 0.97)−5.7, 4.5
A1 volume-A2 volume 0.2 (−0.21, 0.68)−2.5, 3.00 (−0.96, 0.91)−4.8, 4.80.9 (−1.30, 3.02)−6.1, 7.9
A1_c_volume-A2_c_volume −2.8 (−6.03, 0.38)−22.7, 17.10.6 (−1.71, 2.83)−11.1, 12.3−1.8 (−6.57, 2.90)−17.2, 13.5
Inter-reader
A1 diameter-B1 diameter−2 (−3.90, −0.15)−13.6, 9.6−0.4(−1.61, 0.87)−6.8, 6.0−2.1 (−3.13, −1.02)−5.5, 1.3
A1 volume-B1 volume−0.7 (−4.04, 2.57)−21.3, 19.8−1.2 (−5.31, 2.94)−22.4, 20.1−1.2 (−6.02, 3.57)−16.8, 14.3
A1_c_volume-B1_c_volume−6 (−11.52, −0.51)−40.2, 28.2−1.1 (−4.82, 2.55)−20.1, 17.9−6.2 (−9.37, −3.05)−16.5, 4.0
Numbers in brackets are the 95% confidence intervals. A1 diameter-A2 diameter means the intraclass correlation coefficient (ICC) of the longest diameters of the lung nodules between the first (A1) and second (A2) measurements by reader A. A1 volume-A2 volume means the ICC for the 3D semiautomatic volume measurements between the first (A1) and second (A2) measurements by reader A. A1_c_volume-A2_c_volume means the ICC for the calculated volume using the longest diameter between the first (A1) and second (A2) measurements by reader A. B1 is the first measurement by reader B.
Table
Table 6. Mean difference and limits of agreement of different nodule types.
Table 6. Mean difference and limits of agreement of different nodule types.
Measurement ComparisonsIsolatedJuxta-VascularJuxta-Pleural
Mean Difference
% (95% CI)		95% Limits of AgreementMean Difference
% (95%CI)		95% Limits of AgreementMean Difference
% (95% CI)		95% Limits of Agreement
Intra-reader
A1 diameter-A2 diameter *−0.4 (−0.96, 0.26)−3.6, 2.9−1.5 (−3.52, 0.50)−9.7, 6.7−0.1 (−1.07, 0.96)−5.7, 5.6
A1 volume-A2 volume *0.9 (0.026, 1.78)−3.8, 5.6−0.3 (−0.98, 0.37)−3.0, 2.4−0.1 (−0.91, 0.77)−4.7, 4.6
A1_c_volume-A2_c_volume *−1.1 (−2.88, 0.77)−10.8, 8.7−4.5 (−10.41, 1.44)−28.6, 19.6−0.2 (−3.21, 2.87)−17.0, 16.6
Inter-reader
A1 diameter-B1 diameter−2.1 (−3.24, −0.92)−8.3, 4.1−2.6 (−5.33, 0.10)−13.7, 8.4−0.2 (−2.01 1.61)−10.2, 9.8
A1 volume-B1 volume−0.8 (−4.20, 2.69)−19.2, 17.6−2.6 (−7.27, 1.97)−21.4, 16.1−0.2 (−4.21, 3.79)−22.3, 21.9
A1_c_volume-B1_c_volume−6.2 (−9.65, −2.76)−24.6, 12.2−7.7 (−15.66, 0.21)−40, 24.5−0.6 (−5.99, 4.70)−30.2, 28.9
Numbers in brackets are the 95% confidence intervals. * A1 diameter-A2 diameter means the intraclass correlation coefficient (ICC) of the longest diameters of the lung nodules between the first (A1) and second (A2) measurements by reader A. A1 volume-A2 volume means the ICC for the 3D semiautomatic volume measurements between the first (A1) and second (A2) measurements by reader A. A1_c_volume-A2_c_volume means the ICC for the calculated volume using the longest diameter between the first (A1) and second (A2) measurements by reader A. B1 is the first measurement by reader B.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Chen, H.; Huang, H.; Zhang, J.; Wang, X.; Han, M.; Ding, C.; Wang, J. Intra- and Inter-Reader Variations in Lung Nodule Measurements: Influences of Nodule Size, Location, and Observers. Diagnostics 2022, 12, 2319. https://doi.org/10.3390/diagnostics12102319

AMA Style

Chen H, Huang H, Zhang J, Wang X, Han M, Ding C, Wang J. Intra- and Inter-Reader Variations in Lung Nodule Measurements: Influences of Nodule Size, Location, and Observers. Diagnostics. 2022; 12(10):2319. https://doi.org/10.3390/diagnostics12102319

Chicago/Turabian Style

Chen, Hong, Haozhe Huang, Jianye Zhang, Xuexue Wang, Mengyang Han, Chanjun Ding, and Jinhong Wang. 2022. "Intra- and Inter-Reader Variations in Lung Nodule Measurements: Influences of Nodule Size, Location, and Observers" Diagnostics 12, no. 10: 2319. https://doi.org/10.3390/diagnostics12102319

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop