Next Article in Journal
Algorithms for All: Can AI in the Mortgage Market Expand Access to Homeownership?
Next Article in Special Issue
A Time Series Approach to Smart City Transformation: The Problem of Air Pollution in Brescia
Previous Article in Journal
Anthropocentrism and Environmental Wellbeing in AI Ethics Standards: A Scoping Review and Discussion
Previous Article in Special Issue
Applying Few-Shot Learning for In-the-Wild Camera-Trap Species Classification
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
AI
Volume 4
Issue 4
10.3390/ai4040044
ai-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Communication

Can Artificial Intelligence Aid Diagnosis by Teleguided Point-of-Care Ultrasound? A Pilot Study for Evaluating a Novel Computer Algorithm for COVID-19 Diagnosis Using Lung Ultrasound

by
Laith R. Sultan
1,*,
Allison Haertter
2,
Maryam Al-Hasani
3,
George Demiris
4,
Theodore W. Cary
3,
Yale Tung-Chen
5 and
Chandra M. Sehgal
3
1
Department of Radiology, Children’s Hospital of Philadelphia, Philadelphia, PA 19104, USA
2
Radiation Oncology Department, University of Pennsylvania, Philadelphia, PA 19104, USA
3
Ultrasound Research Lab, Department of Radiology, University of Pennsylvania, Philadelphia, PA 19103, USA
4
Informatics Division of the Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA 19104, USA
5
Emergency Medicine Department, La Madrida Hospital, 28006 Madrid, Spain
*
Author to whom correspondence should be addressed.
AI 2023, 4(4), 875-887; https://doi.org/10.3390/ai4040044
Submission received: 11 July 2023 / Revised: 17 September 2023 / Accepted: 22 September 2023 / Published: 10 October 2023
(This article belongs to the Special Issue Feature Papers for AI)
Browse Figures
Review Reports Versions Notes

Abstract

:
With the 2019 coronavirus disease (COVID-19) pandemic, there is an increasing demand for remote monitoring technologies to reduce patient and provider exposure. One field that has an increasing potential is teleguided ultrasound, where telemedicine and point-of-care ultrasound (POCUS) merge to create this new scope. Teleguided POCUS can minimize staff exposure while preserving patient safety and oversight during bedside procedures. In this paper, we propose the use of teleguided POCUS supported by AI technologies for the remote monitoring of COVID-19 patients by non-experienced personnel including self-monitoring by the patients themselves. Our hypothesis is that AI technologies can facilitate the remote monitoring of COVID-19 patients through the utilization of POCUS devices, even when operated by individuals without formal medical training. In pursuit of this goal, we performed a pilot analysis to evaluate the performance of users with different clinical backgrounds using a computer-based system for COVID-19 detection using lung ultrasound. The purpose of the analysis was to emphasize the potential of the proposed AI technology for improving diagnostic performance, especially for users with less experience.

1. Introduction

The coronavirus disease (COVID-19) pandemic has accelerated the adoption of technologies that facilitate patient care while reducing viral spread. Telemedicine is one of the technologies that expanded rapidly during the pandemic across a broad range of clinical practices [1,2,3]. It became the safest interactive system to maintain social distancing between patients and clinicians. One area of telemedicine that has an increasing potential is teleguided ultrasound, where telemedicine and point-of-care ultrasound (POCUS) merge to create this new scope [4,5]. Teleguided POCUS can minimize staff exposure while preserving patient safety and oversight during bedside procedures. As COVID-19 placed unprecedented pressure on healthcare services, care workers responsible for critically ill patients faced significant challenges. To meet such unprecedented demand for clinical services, additional workforce support will likely prove vital. This may involve utilizing other potential healthcare workers alongside the existing staff. These potential new operators include those with alternate ultrasound (US) imaging experience [6,7] and those with no previous US imaging experience [8,9,10,11]. In this paper, we propose a pragmatic consideration of the remote monitoring of COVID-19 patients using teleguided POCUS, supported by AI technologies, by non-experienced personnel including the patients themselves. In pursuit of this goal, we performed a preliminary analysis to evaluate a novel computer algorithm for characterizing COVID-19 using POCUS. The performances of the algorithm users with different experiences ranging from non-expert to experienced user were compared. The primary aim of the tool we investigate in this study, is to demonstrate that similar AI-driven tools have the potential to significantly enhance the automated detection of both normal and abnormal findings, even when utilized by individuals with limited medical experience. Should these AI tools undergo validation in larger-scale studies, they could subsequently be integrated into POCUS devices. This integration would empower less experienced users to detect normal anatomical structures and abnormalities with a greater degree of automation and accuracy.
In essence, our research sets the stage for a promising future in which AI technologies play a pivotal role in enhancing the capabilities of healthcare professionals and even patients themselves in effectively monitoring and managing COVID-19 cases using remote teleguided POCUS.

2. Telemedicine in the Time of the Pandemic

The use of telemedicine during the pandemic for patients’ management provided several additional advantages [1,2,3], such as reduced patient and employee exposure, increased safety and monitoring for patients with pre-existing medical conditions, and increased hospital personnel availability. Remote monitoring also decreased the total cost of care, with upwards of a 32% reduction in the cost of care, due to the shift to a value-based care model [4,5]. Many hospitals began such programs to support patients with COVID-19, in addition to many other conditions such as congestive heart failure, cancer, deep vein thromboses, and infections such as cellulitis and sepsis [12]. The U.S. military’s health system actively used remote monitoring. Brooke Army Medical Center in San Antonio, for example, used a COVID-19 remote monitoring program in which patients are enrolled directly from hospital inpatient units or the emergency department prior to discharge. Kaiser Permanente, the Mayo Clinic, and Intermountain Healthcare in Utah scaled up and expanded their remote monitoring programs, where their virtual hospital is staffed 24/7 by remote monitoring technicians and has tele-nurses, tele-hospitalists, and other tele-advanced practice providers available depending on the patient’s condition. Studies on the remote patient monitoring (RPM) of COVID-19-positive patients reported no fatalities occurring amongst the 244 patients involved [13,14]. These studies showed that RPM programs were able to reduce anxiety due to the uncertainties in, and constantly evolving nature of, guidelines amongst patients who have tested positive or are suspected of having COVID-19 [3,14] by providing patients with guidance on when symptoms are substantial enough to warrant hospital visits [3,13].

3. The Role of POCUS in the Management of COVID-19 Patients

Since its inception for military use in 1998, portable ultrasound or point-of-care ultrasound (POCUS) has grown in accessibility, useability, and technological sophistication [15], revolutionizing the accessibility of diagnostic imaging to resource-limited areas and emergency situations [15,16,17]. POCUS use has increased significantly during the pandemic for monitoring the inflammatory lung changes seen in COVID-19 patients [18,19,20,21,22,23,24]. This is facilitated in part by the typical sonographic characteristics of COVID-19-associated lung injury during disease progression and recovery [25,26,27,28,29,30,31,32,33,34,35,36,37]. Usually, POCUS is operated by experienced clinicians. The primary POCUS imaging technique is based on brightness mode (B-mode) imaging of the pleural lining and lung tissue [38,39,40,41,42,43]. Under normal circumstances, the tissue at the boundary between air and soft tissue or fluids is very reflective, and tissues within the images are indistinguishable [42,43]; hence, with a highly aerated lung surface, ultrasound penetration beyond the pleural line is hindered. When the alveoli fill with fluid, which frequently occurs during infections, the interface becomes penetrable by ultrasound waves generating bright vertical artifacts called B-lines. Despite being well-described and utilized in clinical practice, the exact underlying cause of these vertical artifacts in the lungs remains unknown [42]. The cause of this phenomenon was initially attributed to reverberation events that occur when the ultrasound beam interacts with thickened interlobular septa; this is known as the “septal hypothesis”. However, subsequent evidence has shown that this hypothesis is at least incomplete, as these artifacts have been observed in naturally collapsed healthy lungs and in synthetic phantoms. Yet, they do provide qualitative “visual” information about a range of pathologies from a clinical perspective [39], and the presence of dense multiple B-lines consistently corresponds to ground glass opacities (GGOs) seen on computed tomography (CT) scans, where GGOs represent a tomographic variation in the air/tissue ratio.
While POCUS technology has seen great advancements in accessibility, the issue that persists in both developing and developed areas of the world is a shortage of radiologists and sonographers who are able to perform the ultrasound imaging. It is this shortage of trained professionals that is driving the push for teleguided ultrasound useability.

4. Teleguided POCUS for Remote Monitoring of COVID-19 Patients

The feasibility of teleguided POCUS has been shown in several studies, the scope of which includes obstetrics [4,5,44,45] and abdominal, thoracic and lung, and trauma situations. Different techniques of teleguided US have been studied, including telerobotic and remote monitoring, with both types being deemed successful implementations regarding adequate image acquisition for diagnostic requirements, user comfort, patient acceptance, and increased accessibility to patients and medical systems.
Earlier works have already discussed the advantages, feasibility, and success of remote COVID-19 patient care, as well as the ability to monitor COVID-19 patients with POCUS. Studies have put forth specific guidelines to facilitate the potential implementation of lung ultrasound (LUS) in telemedicine applications. These guidelines concern the medical personnel who might require formal training and qualification, as well as the adoption of specific protocols for focused assessment [46]. Pivetta et al. (2020) [44] have already shown that an ultrasound-experienced individual can perform remote self-administration of POCUS with the aid of teleguidance. While self-performed ultrasound may sound improbable, it has already been performed, supporting the feasibility of the proposed model. There exists a report about a nurse who was exposed to COVID-19 and completed 16 days of self-performed POCUS to monitor their health progression with widely adapted protocols [46]. In general, the image quality achieved during self-performance was greatly enhanced once teleguidance was added to provide feedback on the imaging protocols used [47]. The study showed that self-performed POCUS is not only possible but can be successful in the detection of changes related to COVID-19 with ultrasound technology and teleguidance support. While this study was carried out by a nurse who had ultrasound imaging experience, there is evidence that novice users can produce diagnosable quality images with only a small amount of training [48,49,50]. The remote-monitored patients can be given a remotely administered didactic and practice session (or sessions) to provide instructions on how to correctly hold and use the ultrasound transducer and the locations in which they are to be targeting for imaging. POCUS scoring is in accordance with the protocol laid out by Soldati et al. (2020) [51].
Our proposed model is centered around teleguided POCUS for the remote monitoring of lung features specific to COVID-19. The aim is to enable untrained staff or even the patients themselves to perform the ultrasound scans under remote guidance. The largest limitation that can be foreseen is patient compliance, specifically amongst patients in the remotely monitored subgroup, who may be unwilling to self-perform daily POCUS imaging due to high discomfort levels, a fear of inadequate care, or the misunderstanding of study protocols. Another potential concern is related to data protection in respect of ultrasound imaging and other acquired health records. This issue has been raised by other authors [5,50], and patients self-performing imaging could be subject to even higher risks.
The primary objective of this paper is to advance the proposed model by evaluating the practicality of novice users independently conducting daily POCUS scans, augmented by a computer-based analysis tool. This innovative approach seeks to address the inherent challenges associated with operator dependency and inconsistencies in imaging settings. By offering standardization in image acquisition, the computer-based tool aims to enhance the dependability and uniformity of POCUS outcomes. Ultimately, the implementation of such a system has the potential to refine the accuracy of lung-feature monitoring for individuals affected by COVID-19. This advancement would not only increase the accessibility of POCUS but also reduce the need for highly trained personnel to be present during the scanning process. By empowering untrained staff or even patients themselves to perform POCUS scans under remote supervision, this model strives to improve the efficiency and effectiveness of remote monitoring, contributing to better healthcare outcomes for COVID-19 patients.

5. AI Can Improve Teleguided POCUS Monitoring of COVID-19 Patients

For the thousands of patients fighting for their lives against this deadly disease and the health care providers who incur a constant risk of infection, the implementation of AI in healthcare can be a game changer. In particular, the use of AI applications can facilitate the teleguided POCUS monitoring of COVID-19 patients. The AI industry, as well as academia, has been developing algorithms for the detection of pathological changes related to COVID-19 [52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68]. Several image-based AI diagnostics have been built to detect COVID-19 changes by using different modes such as CT scans, X-rays, cough sounds, MRIs, and ultrasound, as well as clinical markers.
Implementing such tools in teleguided ultrasound would have the potential to facilitate remote testing and expand access. There are two ways in which AI can have an impact. The first is automated image analysis, particularly for people with less familiarity with imaging, to facilitate the effective use of the scanner. Users who are not experienced in lung imaging could progress to screening patients with high accuracy. Second, by providing real-time instructions and feedback through telemedicine apps, the system’s AI analyzes live image quality and gives suggestions on how to move and position the ultrasound’s transducer to improve clarity. This can help more front-line hospital staff conduct examinations on their own, reducing exposures of sonography personnel. Philips Lumify and Butterfly IQ, the largest handheld device manufacturers, introduced a tele-ultrasound system that streams the images live over a video call or over a smartphone or tablet.
Preliminary analysis: We have introduced a computer-based system that captures changing patterns in pleural lines associated with COVID-19 by means of a quantitative analysis of lung ultrasound [69]. The analysis only requires a minimal input from the user, who selects a seed region and then validates the segmentation Figure 1A. The use of the algorithm requires minimal training. Quantitative lung features describing pleural line thickness and margin morphology including tortuosity, projected intensity deviation, and nonlinearity were evaluated Figure 1B. The use of the algorithm facilitated high accuracy in detecting COVID-19 and improved the diagnostic performance of LUS Figure 2.
In the current study, we compared the performance of five users with different experiences ranging from no clinical experience to a highly experienced user to analyze the ultrasound images using the algorithm. The users’ backgrounds are summarized in Table 1.
The images were received for analysis anonymized and without patient-related information. Of twenty B-mode ultrasound images, ten images were acquired from COVID-19 patients, and another ten independent images were acquired from normal cases. The images were acquired using a portable ultrasound device (Butterfly IQ) using a specially designed lung-imaging setting. The lung preset has been optimized for demonstration and sliding of the pleural surface, as well as A-lines and B-lines throughout the 15 cm field. The preset transforms from a curvilinear array to a linear array when the depth is reduced to 7 cm or less. This optimizes near-field lung sliding dynamics as well as pleural artifacts. The same settings were used for acquiring all images used in the analysis. The patients’ informed consent for performing image analysis on anonymized images was exempted by the University of Pennsylvania’s institutional review board. In respect of the imaging data, all the methods employed were in accordance with relevant guidelines and regulations. The experimental protocol of the retrospective study was approved by the University of Pennsylvania’s institutional review board. We performed a quantitative analysis that utilized image segmentation and feature extraction using IDL software (version 8.5, L3Harris Geospatial, Boulder, CO, USA) [69]. Training and guidance in image analysis were provided by the user with the highest experience to other users. To segment the pleural lines, we developed a semi-automatic segmentation tool called a “wand”. This tool utilizes a basic form of “region growing”, where the algorithm automatically grows the region by selecting pixel seeds with similar grayscale values within a tolerance range of ±10 gray levels (default value). The user’s input is minimal, requiring only clicking and validating the segmentation and making corrections when necessary. After detecting the pleural line, the software extracts quantitative features that describe the depth (thickness), margin morphology, brightness, and heterogeneity [69,70]. The thickness parameters measure the change in the pleural line’s horizontal depth. Margin morphology features, including tortuosity, projected intensity deviation (PID), and nonlinearity, measure irregularities in the pleural line’s margin shape.
The results for the different users are shown in Table 2. The AUC (area under the ROC curve) as a metric for binary classification models based on the ROC (receiver operating characteristic) curve was used for the discrimination of the groups, normal vs. COVID-19. A strong performance was observed for the different users with the AUC ranging between 0.84 and 0.99. For users 1, 2, and 5, who had long experience with LUS imaging, the results showed that five features from seven were significantly different between COVID-19 and normal, (p < 0.05). The performance for these users ranged between 0.96 and 0.99. For user 3, who was a student familiar somewhat with LUS, the results showed that five features were also significant between normal and COVID-19 cases with a performance of the AUC of 0.93. For user 4, who was the least experienced user, the performance was respectable, with the AUC being 0.84.
Using the computer-based analysis tool improved the consistency of the readers’ performance. On average, the interclass agreement was 0.83 ± 0.12, indicating excellent agreement. In comparison, the interclass correlation coefficient (ICC) for clinical assessment without the algorithm was 0.77, indicating good agreement. The agreement between readers was comparable and even surpassed the ICC without the algorithm. These results align with the previous literature, where AI algorithms achieved results that are comparable to human performance [41].
The automated segmentation methods that were performed in this study can be easily integrated into the operation of a scanner for real-time bedside assessment. In addition, fully automated machine learning segmentation could be optimized and performed. This technology, when implemented successfully in clinical practice, will increase confidence in diagnosis, especially in low-resource communities around the globe that lack experience in lung ultrasound. Automated systems for lung ultrasound data analysis are an active area of research, and there have been several recent advancements in this field. The automated segmentation of the pleural line is a crucial step in the automated analysis of LUS images, as it provides accurate measurements of pleural line changes, which are important indicators of lung pathology. There have been few recent studies that have focused on developing automated segmentation methods for the pleural line in lung ultrasound images. One common approach is to use deep learning algorithms, such as convolutional neural networks (CNNs), to segment the pleural line automatically. For example, a study published in 2020 proposed a fully automated deep learning algorithm for segmenting the pleural line in LUS images [71]. The algorithm used a U-Net architecture, a type of CNN commonly used for image segmentation, and achieved high accuracy in segmenting the pleural line. Another study published in 2021 proposed a segmentation method based on the combination of a deep learning algorithm and an active contour model [72]. The proposed method achieved high accuracy in segmenting the pleural line in both healthy and diseased lungs and demonstrated promising results in a clinical setting. Furthermore, there have also been studies that have focused on developing methods for correcting errors in automated pleural line segmentation. For example, a study published in 2020 proposed a method for correcting errors in automated pleural line segmentation using a graph-based approach [73]. Overall, these studies demonstrate that automated segmentation of the pleural line using deep learning algorithms is a promising approach for analyzing LUS images. In our studies, we evaluated a simpler method of segmentation and feature extraction that can be easily implemented in clinical settings.
The present study has certain limitations, primarily due to its retrospective nature and the small sample size of the image data. To establish the method’s validity, future prospective studies with larger datasets are necessary. Additionally, the variability resulting from the selection of a single image from the clip can be addressed through advancements in automated segmentation and region selection techniques, offering more accurate and reliable results. Further research is warranted to develop improved methods for analyzing the pleural line, particularly considering the functional aspects such as lung sliding motion, in addition to structural changes. Incorporating features that capture both functional and structural aspects could enhance disease detection. Furthermore, to enhance the safety and effectiveness of AI tools for remote monitoring with POCUS, several critical safeguards must be in place. These include rigorous validation and testing to ensure the system’s accuracy, human oversight to review and confirm AI-generated findings, continuous monitoring and feedback mechanisms for ongoing improvement, comprehensive education and training for users with limited experience, clear communication of the AI tool’s confidence levels, well-defined protocols for risk mitigation in cases of false positives or negatives, accessible user support channels, regular system updates, established lines of accountability, and a commitment to ethical considerations that prioritize patient well-being above all else. These safeguards collectively serve to mitigate the risks associated with false positives and false negatives, fostering trust in AI-driven healthcare solutions while prioritizing patient safety.
In developing the proposed system for remote monitoring with POCUS, valuable lessons are learned from existing self-monitoring modalities such as blood-sugar and blood-pressure monitoring. Key principles include designing a user-friendly interface with clear instructions, automating data capture to minimize user error, providing real-time feedback and data visualization, implementing alerts and notifications for critical findings, emphasizing user education and training, integrating remote monitoring and telehealth capabilities, prioritizing data privacy and security, ensuring accessibility for all users, and maintaining a feedback-driven approach for continuous improvement. By incorporating these lessons, the authors can enhance the usability, effectiveness, and safety of their system, empowering patients with limited experience to actively participate in their healthcare while maintaining data accuracy and reliability.
Future opportunities for AI: Another form of AI that holds incredible promise for healthcare involves augmented and virtual reality (AR/VR) technology [74,75,76,77]. Both are quickly bringing forth possibilities that were never previously imagined. The concept of the metaverse, for example, has great potential for an entirely new approach to enhance telehealth solutions by reducing the distance between the patients and their providers. It can elevate virtual care from a 2D to a 3D experience, changing the way patients interact with healthcare providers in a fundamental way. Facebook’s recent transformation to “Meta” is an ode to the company’s foresight and vision that the metaverse is the next frontier in technological innovation. Another example is Microsoft Mesh, a “holoportation” and mixed reality platform that aims to make digital connections lifelike and enable new ways to remotely teach, learn, and perform tasks virtually. This comes in line with Microsoft’s Hololens technology that has been investigated for efficacy in remote medical care in surgical and nonsurgical case studies. Companies are also investing in augmenting the telemedicine experience through AR/VR for more friendly and safe patient experiences.
In recent studies focused on COVID-19 diagnosis, researchers have explored the potential of VR and AR immersive technologies [78,79,80]. These investigations have sought to integrate CT imaging with VR technology to create an automated system capable of navigating 3D visualizations. The objective has been to develop a VR platform specifically designed for visualizing and interacting with 3D representations of the lungs and infected regions. By leveraging AR/VR technology, it is envisioned that remote monitoring of COVID-19 patients with POCUS can be further facilitated, particularly in terms of education, training, and guidance for patients and other non-trained personnel.

6. Conclusions

Remote monitoring of COVID-19 patients in the comfort of their homes holds significant promise, made feasible through the integration of AI-supported teleguidance systems. This innovative approach POCUS technology, even in the hands of untrained individuals, including the patients themselves. In this paper, we have elucidated the mechanisms for effectively training patients to adeptly utilize teleguided portable ultrasound devices for self-monitoring and reporting changes pertaining to the progression of the disease. Our preliminary analysis has yielded encouraging results, revealing that individuals with limited experience can achieve a remarkable level of accuracy in detecting COVID-19 when assisted by AI. The implications of this breakthrough are profound; if implemented successfully on a broader scale, it stands to substantially reduce the exposure of healthcare workers to the virus and curtail the risk of unnecessary disease transmission. In essence, this AI-driven telemonitoring system not only empowers patients to actively participate in managing their health but also serves as a vital tool in our collective efforts to mitigate the spread of COVID-19, ultimately safeguarding public health and healthcare resources.

Author Contributions

L.R.S., A.H. and G.D. conceptualized the study. Y.T.-C. acquired the data. L.R.S., M.A.-H., T.W.C. and C.M.S. analyzed and interpreted the data. L.R.S. and A.H. wrote the manuscript. This article contains original data. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study did not require ethical approval. The LUS images were received anonymized for analysis with no patient information.

Informed Consent Statement

Not applicable.

Data Availability Statement

The datasets generated and/or analyzed during the current study are not publicly available due the ongoing nature of the research study but are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Annis, T.; Pleasants, S.; Hultman, G.; Lindemann, E.; Thompson, J.A.; Billecke, S.; Badlani, S.; Melton, G.B. Rapid implementation of a COVID-19 remote patient monitoring program. J. Am. Med. Inform. Assoc. 2020, 27, 1326–1330. [Google Scholar] [CrossRef] [PubMed]
  2. Silberzahn, T. Remote Patient Monitoring for Safe and Effective Management of COVID-19 Patients. HealthManagement.org J. 2020, 20, 513–514. [Google Scholar]
  3. Tabacof, L.; Kellner, C.; Breyman, E.; Dewil, S.; Braren, S.; Nasr, L.; Tosto, J.; Cortes, M.; Putrino, D. Remote Patient Monitoring for Home Management of Coronavirus Disease 2019 in New York: A Cross-Sectional Observational Study. Telemed. E-Health 2021, 27, 641–648. [Google Scholar] [CrossRef] [PubMed]
  4. Britton, N.; Miller, M.A.; Safadi, S.; Siegel, A.; Levine, A.R.; McCurdy, M.T. Tele-ultrasound in resource-limited settings: A systematic review. Front. Public Health 2019, 7, 244. [Google Scholar] [CrossRef] [PubMed]
  5. Salerno, A.; Tupchong, K.; Verceles, A.C.; Mccurdy, M.T. Point-of-Care Teleultrasound: A Systematic Review. Telemed. E-Health 2020, 26, 1314–1321. [Google Scholar] [CrossRef] [PubMed]
  6. Buonsenso, D.; Moro, F.; Inchingolo, R.; Smargiassi, A.; Demi, L.; Soldati, G.; Moroni, R.; Lanzone, A.; Scambia, G.; Testa, A.C. Effectiveness of rapid lung ultrasound training program for gynecologists and obstetricians managing pregnant women with suspected COVID-19. Ultrasound Obstet. Gynecol. 2020, 56, 110–111. [Google Scholar] [CrossRef]
  7. Vigotti, F.N.; Di Benedetto, C.; Fop, F.; Bianco, S.; Bilucaglia, D.; Cesano, G. Lung ultrasonography performed by nephrologist: COVID-19 as an opportunity to reveal ultrasound’s full potential and usefulness in the dialysis room. Clin. Kidney J. 2022, 16, 541–548. [Google Scholar] [CrossRef]
  8. Marini, T.J.; Kaproth-Joslin, K.; Ambrosini, R.; Baran, T.M.; Dozier, A.M.; Zhao, Y.T.; Satheesh, M.; Mahony Reátegui-Rivera, C.; Sifuentes, W.; Rios-Mayhua, G.; et al. Volume sweep imaging lung teleultrasound for detection of COVID-19 in Peru: A multicentre pilot study. BMJ Open 2022, 12, e061332. [Google Scholar] [CrossRef]
  9. Kirkpatrick, A.W.; McKee, I.; McKee, J.L.; Ma, I.; McBeth, P.B.; Roberts, D.J.; Wurster, C.L.; Parfitt, R.; Ball, C.G.; Oberg, S.; et al. Remote just-in-time telementored trauma ultrasound: A double-factorial randomized controlled trial examining fluid detection and remote knobology control through an ultrasound graphic user interface display. Am. J. Surg. 2016, 211, 894–902.e1. [Google Scholar] [CrossRef]
  10. McBeth, P.B.; Crawford, I.; Blaivas, M.; Hamilton, T.; Musselwhite, K.; Panebianco, N.; Melniker, L.; Ball, C.G.; Gargani, L.; Gherdovich, C.; et al. Simple, almost anywhere, with almost anyone: Remote low-cost telementored resuscitative lung ultrasound. J. Trauma 2011, 71, 1528–1535. [Google Scholar] [CrossRef]
  11. Raiteri, A.; Muratori, L.; Faggiano, C.; Alvisi, M.; Serio, I.; Piscaglia, F. Efficacy of a short course of lung ultrasound for primary care physicians in the assessment of COVID-19-positive patients. Fam. Pract. 2022, 39, 656–661. [Google Scholar] [CrossRef] [PubMed]
  12. Friedman, A.B.; Gervasi, S.; Song, H.; Bond, A.M.; Chen, A.T.; Bergman, A.; David, G.; Bailey, J.M.; Brooks, R.; Smith-McLallen, A. Telemedicine Catches on: Changes in the Utilization of Telemedicine Services during the COVID-19 Pandemic. Am. J. Manag. Care 2022, 28, e1–e6. [Google Scholar] [PubMed]
  13. Thornton, J. The “virtual wards” supporting patients with covid-19 in the community. BMJ 2020, 369, 1–2. [Google Scholar] [CrossRef] [PubMed]
  14. Khawam, E.; Khouli, H.; Pozuelo, L. Treating acute anxiety in patients with COVID-19: Posted April 26, 2020. Clevel. Clin. J. Med. 2020, 87, 1–4. [Google Scholar] [CrossRef]
  15. Lee, L.; DeCara, J.M. Point-of-Care Ultrasound. Curr. Cardiol. Rep. 2020, 22, 149. [Google Scholar] [CrossRef]
  16. Miles, R.C.; Onega, T.; Lee, C.I. Addressing Potential Health Disparities in the Adoption of Advanced Breast Imaging Technologies. Acad. Radiol. 2018, 25, 547–551. [Google Scholar] [CrossRef]
  17. Smith, A.; Addison, R.; Rogers, P.; Stone-McLean, J.; Boyd, S.; Hoover, K.; Pollard, M.; Dubrowski, A.; Parsons, M. Remote mentoring of point-of-care ultrasound skills to inexperienced operators using multiple telemedicine platforms: Is a cell phone good enough? J. Ultrasound Med. 2018, 37, 2517–2525. [Google Scholar] [CrossRef]
  18. Hussain, A.; Via, G.; Melniker, L.; Goffi, A.; Tavazzi, G.; Neri, L.; Villen, T.; Hoppmann, R.; Mojoli, F.; Noble, V.; et al. Multi-organ point-of-care ultrasound for COVID-19 (PoCUS4COVID): International expert consensus. Crit. Care 2020, 24, 702. [Google Scholar] [CrossRef]
  19. Demi, L.; Mento, F.; Di Sabatino, A.; Fiengo, A.; Sabatini, U.; Macioce, V.N.; Robol, M.; Tursi, F.; Sofia, C.; Di Cienzo, C.; et al. Lung Ultrasound in COVID-19 and Post-COVID-19 Patients, an Evidence-Based Approach. J. Ultrasound Med. 2022, 41, 2203–2215. [Google Scholar] [CrossRef]
  20. Sultan, L.R.; Sehgal, C.M. A Review of Early Experience in Lung Ultrasound in the Diagnosis and Management of COVID-19. Ultrasound Med. Biol. 2020, 46, 2530–2545. [Google Scholar] [CrossRef]
  21. Soldati, G.; Smargiassi, A.; Perrone, T.; Torri, E.; Mento, F.; Demi, L.; Inchingolo, R. There is a Validated Acquisition Protocol for Lung Ultrasonography in COVID-19 Pneumonia. J. Ultrasound Med. 2021, 40, 2783. [Google Scholar] [CrossRef] [PubMed]
  22. Perrone, T.; Soldati, G.; Padovini, L.; Fiengo, A.; Lettieri, G.; Sabatini, U.; Gori, G.; Lepore, F.; Garolfi, M.; Palumbo, I.; et al. A New Lung Ultrasound Protocol Able to Predict Worsening in Patients Affected by Severe Acute Respiratory Syndrome Coronavirus 2 Pneumonia. J. Ultrasound Med. 2021, 40, 1627–1635. [Google Scholar] [CrossRef] [PubMed]
  23. Smargiassi, A.; Soldati, G.; Borghetti, A.; Scoppettuolo, G.; Tamburrini, E.; Testa, A.C.; Moro, F.; Natale, L.; Larici, A.R.; Buonsenso, D.; et al. Lung ultrasonography for early management of patients with respiratory symptoms during COVID-19 pandemic. J. Ultrasound 2020, 23, 449–456. [Google Scholar] [CrossRef] [PubMed]
  24. Soldati, G.; Smargiassi, A.; Inchingolo, R.; Buonsenso, D.; Perrone, T.; Briganti, D.F.; Perlini, S.; Torri, E.; Mariani, A.; Mossolani, E.E.; et al. Proposal for International Standardization of the Use of Lung Ultrasound for Patients With COVID-19: A Simple, Quantitative, Reproducible Method. J. Ultrasound Med. 2020, 39, 1413–1419. [Google Scholar] [CrossRef] [PubMed]
  25. Qian, X.; Wodnicki, R.; Kang, H.; Zhang, J.; Tchelepi, H.; Zhou, Q. Current Ultrasound Technologies and Instrumentation in the Assessment and Monitoring of COVID-19 Positive Patients. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 2020, 67, 2230–2240. [Google Scholar] [CrossRef] [PubMed]
  26. Orosz, G.; Gyombolai, P.; Tóth, J.T.; Szabó, M. Reliability and clinical correlations of semi-quantitative lung ultrasound on BLUE points in COVID-19 mechanically ventilated patients: The ‘BLUE-LUSS’-A feasibility clinical study. PLoS ONE 2022, 17, e0276213. [Google Scholar] [CrossRef]
  27. Gil-Rodríguez, J.; Martos-Ruiz, M.; Benavente-Fernández, A.; Aranda-Laserna, P.; Montero-Alonso, M.Á.; Peregrina-Rivas, J.A.; Fernández-Reyes, D.; Martínez de Victoria-Carazo, J.; Guirao-Arrabal, E.; Hernández-Quero, J. Lung ultrasound score severity cut-off points in COVID-19 pneumonia. A systematic review and validating cohort. Med. Clin. 2023, 160, 531–539. [Google Scholar] [CrossRef]
  28. Blazic, I.; Cogliati, C.; Flor, N.; Frija, G.; Kawooya, M.; Umbrello, M.; Ali, S.; Baranne, M.L.; Cho, Y.J.; Pitcher, R.; et al. The use of lung ultrasound in COVID-19. ERJ Open Res. 2023, 9, 00196-2022. [Google Scholar] [CrossRef]
  29. Deng, Q.; Zhang, Y.; Wang, H.; Chen, L.; Yang, Z.; Peng, Z.; Liu, Y.; Feng, C.; Huang, X.; Jiang, N.; et al. Semiquantitative lung ultrasound scores in the evaluation and follow-up of critically ill patients with COVID-19: A single-center study. Acad. Radiol. 2020, 27, 1363–1372. [Google Scholar] [CrossRef]
  30. Bianchi, S.; Savinelli, C.; Paolucci, E.; Pelagatti, L.; Sibona, E.; Fersini, N.; Buggea, M.; Tozzi, C.; Allescia, G.; Paolini, D.; et al. Point-of-care ultrasound (PoCUS) in the early diagnosis of novel coronavirus 2019 disease (COVID-19) in a first-level emergency department during a SARS-CoV-2 outbreak in Italy: A real-life analysis. Intern. Emerg. Med. 2022, 17, 193–204. [Google Scholar] [CrossRef]
  31. Ji, L.; Cao, C.; Gao, Y.; Zhang, W.; Xie, Y.; Duan, Y.; Kong, S.; You, M.; Ma, R.; Jiang, L.; et al. Prognostic value of bedside lung ultrasound score in patients with COVID-19. Crit. Care 2020, 24, 700. [Google Scholar] [CrossRef] [PubMed]
  32. Brahier, T.; Meuwly, J.Y.; Pantet, O.; Brochu Vez, M.J.; Gerhard Donnet, H.; Hartley, M.A.; Hugli, O.; Boillat-Blanco, N. Lung Ultrasonography for Risk Stratification in Patients with Coronavirus Disease 2019 (COVID-19): A Prospective Observational Cohort Study. Clin. Infect. Dis. 2021, 73, e4189–e4196. [Google Scholar] [CrossRef] [PubMed]
  33. Schaad, S.; Brahier, T.; Hartley, M.A.; Cordonnier, J.B.; Bosso, L.; Espejo, T.; Pantet, O.; Hugli, O.; Carron, P.N.; Meuwly, J.Y.; et al. Point-of-care lung ultrasonography for early identification of mild COVID-19: A prospective cohort of outpatients in a Swiss screening center. BMJ Open 2022, 12, e060181. [Google Scholar] [CrossRef] [PubMed]
  34. Zanforlin, A.; Strapazzon, G.; Falk, M.; Gallina, V.; Viteritti, A.; Valzolgher, L.; La Guardia, M.; Ferro, F.; Pagani, L.; Vezzali, N. Lung Ultrasound in the Emergency Department for Early Identification of COVID-19 Pneumonia. Respiration 2021, 100, 145–153. [Google Scholar] [CrossRef] [PubMed]
  35. Di Gioia, C.C.; Artusi, N.; Xotta, G.; Bonsano, M.; Sisto, U.G.; Tecchiolli, M.; Orso, D.; Cominotto, F.; Amore, G.; Meduri, S.; et al. Lung ultrasound in ruling out COVID-19 pneumonia in the ED: A multicentre prospective sensitivity study. Emerg. Med. J. 2022, 39, 199–205. [Google Scholar] [CrossRef]
  36. Lichter, Y.; Topilsky, Y.; Taieb, P.; Banai, A.; Hochstadt, A.; Merdler, I.; Gal Oz, A.; Vine, J.; Goren, O.; Cohen, B.; et al. Lung ultrasound predicts clinical course and outcomes in COVID-19 patients. Intensive Care Med. 2020, 46, 1873–1883, Erratum in Intensive Care Med. 2020, 46, 2128–2129. [Google Scholar] [CrossRef]
  37. Tombini, V.; Di Capua, M.; Capsoni, N.; Lazzati, A.; Bergamaschi, M.; Gheda, S.; Ghezzi, L.; Cassano, G.; Albertini, V.; Porta, L.; et al. Risk Stratification in COVID-19 Pneumonia—Determining the Role of Lung Ultrasound. Ultraschall Med. 2022, 43, 168–176. [Google Scholar] [CrossRef]
  38. Peschiera, E.; Mento, F.; Demi, L. Numerical study on lung ultrasound B-line formation as a function of imaging frequency and alveolar geometries. J. Acoust. Soc. Am. 2021, 149, 2304. [Google Scholar] [CrossRef]
  39. Mento, F.; Demi, L. On the influence of imaging parameters on lung ultrasound B-line artifacts, in vitro study. J. Acoust. Soc. Am. 2020, 148, 975. [Google Scholar] [CrossRef]
  40. Soldati, G.; Demi, M.; Inchingolo, R.; Smargiassi, A.; Demi, L. On the Physical Basis of Pulmonary Sonographic Interstitial Syndrome. J. Ultrasound Med. 2016, 35, 2075–2086. [Google Scholar] [CrossRef]
  41. Mento, F.; Khan, U.; Faita, F.; Smargiassi, A.; Inchingolo, R.; Perrone, T.; Demi, L. State of the Art in Lung Ultrasound, Shifting from Qualitative to Quantitative Analyses. Ultrasound Med. Biol. 2022, 48, 2398–2416. [Google Scholar] [CrossRef] [PubMed]
  42. Demi, L.; Demi, M.; Smargiassi, A.; Inchingolo, R.; Faita, F.; Soldati, G. Ultrasonography in lung pathologies: New perspectives. Multidiscip. Respir. Med. 2014, 9, 27. [Google Scholar] [CrossRef] [PubMed]
  43. Mento, F.; Demi, L. Dependence of lung ultrasound vertical artifacts on frequency, bandwidth, focus and angle of incidence: An in vitro study. J. Acoust. Soc. Am. 2021, 150, 4075. [Google Scholar] [CrossRef] [PubMed]
  44. Pivetta, E.; Girard, E.; Locascio, F.; Lupia, E.; Martin, J.D.; Stone, M. Self-Performed Lung Ultrasound for Home Monitoring of a Patient Positive for Coronavirus Disease 2019. Chest J. 2020, 158, E93–E97. [Google Scholar] [CrossRef] [PubMed]
  45. Adams, S.J.; Burbridge, B.E.; Badea, A.; Kanigan, N.; Bustamante, L.; Babyn, P.; Mendez, I. A cross over comparison of standard and telerobotic approaches to prenatal sonography. J. Ultrasound Med. 2018, 37, 2603–2612. [Google Scholar] [CrossRef]
  46. Douglas, T.M.; Levine, A.R.; Olivieri, P.P.; McCurdy, M.T.; Papali, A.; Zubrow, M.T.; Rodick, K.M.; Hurley, J.M.; Verceles, A.C. Brief training increases nurses’ comfort using tele-ultrasound: A feasibility study. Intensive Crit. Care Nurs. 2019, 51, 45–49. [Google Scholar] [CrossRef]
  47. Demi, L.; Wolfram, F.; Klersy, C.; De Silvestri, A.; Ferretti, V.V.; Muller, M.; Miller, D.; Feletti, F.; Wełnicki, M.; Buda, N.; et al. New International Guidelines and Consensus on the use of Lung Ultrasound. J. Ultrasound Med. 2022, 42, 309–344. [Google Scholar] [CrossRef]
  48. Dini, F.L.; Bergamini, C.; Allegrini, A.; Scopelliti, M.; Secco, G.; Miccoli, M.; Boni, S.; Brigada, R.; Perlini, S. Bedside wireless lung ultrasound for the evaluation of COVID-19 lung injury in senior nursing home residents. Monaldi Arch. Chest Dis. 2020, 90, 523–527. [Google Scholar] [CrossRef]
  49. Levine, A.R.; McCurdy, M.T.; Zubrow, M.T.; Papali, A.; Mallemat, H.A.; Verceles, A.C. Tele-intensivists can instruct non-physicians to acquire high-quality ultrasound images. J. Crit. Care 2015, 30, 871–875. [Google Scholar] [CrossRef]
  50. Robertson, T.E.; Levine, A.R.; Verceles, A.C.; Buchner, J.A.; Lantry, J.H.; Papali, A.; Zubrow, M.T.; Colas, L.N.; Augustin, M.E.; McCurdy, M.T. Remote tele-mentored ultrasound for non-physician learners using FaceTime: A feasibility study in a low-income country. J. Crit. Care 2017, 40, 145–148. [Google Scholar] [CrossRef]
  51. Lieveld, A.W.; Kok, B.; Azijli, K.; Schuit, F.H.; van de Ven, P.M.; de Korte, C.L.; Nijveldt, R.; van den Heuvel, F.M.; Teunissen, B.P.; Hoefsloot, W.; et al. Assessing COVID-19 pneumonia—clinical extension and risk with point-of-care ultrasound: A multicenter, prospective, observational study. J. Am. Coll. Emerg. Physicians Open 2021, 2, e12429. [Google Scholar] [CrossRef] [PubMed]
  52. Shi, F.; Wang, J.; Shi, J.; Wu, Z.; Wang, Q.; Tang, Z.; He, K.; Shi, Y.; Shen, D. Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19. IEEE Rev. Biomed. Eng. 2021, 14, 4–15. [Google Scholar] [CrossRef] [PubMed]
  53. Syeda, H.B.; Syed, M.; Sexton, K.W.; Syed, S.; Begum, S.; Syed, F.; Prior, F.; Yu, F., Jr. Role of Machine Learning Techniques to Tackle the COVID-19 Crisis: Systematic Review. JMIR Med. Inform. 2021, 9, e23811. [Google Scholar] [CrossRef]
  54. Khanna, V.V.; Chadaga, K.; Sampathila, N.; Prabhu, S.; Chadaga, R.; Umakanth, S. Diagnosing COVID-19 using artificial intelligence: A comprehensive review. Netw. Model. Anal. Health Inform. Bioinform. 2022, 11, 25. [Google Scholar] [CrossRef]
  55. Piccialli, F.; di Cola, V.S.; Giampaolo, F.; Cuomo, S. The Role of Artificial Intelligence in Fighting the COVID-19 Pandemic. Inf. Syst. Front. 2021, 23, 1467–1497. [Google Scholar] [CrossRef] [PubMed]
  56. Fatima, N.; Mento, F.; Zanforlin, A.; Smargiassi, A.; Torri, E.; Perrone, T.; Demi, L. Human-to-AI Interrater Agreement for Lung Ultrasound Scoring in COVID-19 Patients. J. Ultrasound Med. 2023, 42, 843–851. [Google Scholar] [CrossRef]
  57. Khan, U.; Afrakhteh, S.; Mento, F.; Fatima, N.; De Rosa, L.; Custode, L.L.; Azam, Z.; Torri, E.; Soldati, G.; Tursi, F.; et al. Benchmark methodological approach for the application of artificial intelligence to lung ultrasound data from COVID-19 patients: From frame to prognostic-level. Ultrasonics 2023, 132, 106994. [Google Scholar] [CrossRef]
  58. Custode, L.L.; Mento, F.; Tursi, F.; Smargiassi, A.; Inchingolo, R.; Perrone, T.; Demi, L.; Iacca, G. Multi-objective automatic analysis of lung ultrasound data from COVID-19 patients by means of deep learning and decision trees. Appl. Soft Comput. 2023, 133, 109926. [Google Scholar] [CrossRef]
  59. Khan, U.; Mento, F.; Nicolussi Giacomaz, L.; Trevisan, R.; Smargiassi, A.; Inchingolo, R.; Perrone, T.; Demi, L. Deep Learning-Based Classification of Reduced Lung Ultrasound Data From COVID-19 Patients. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 2022, 69, 1661–1669. [Google Scholar] [CrossRef]
  60. Frank, O.; Schipper, N.; Vaturi, M.; Soldati, G.; Smargiassi, A.; Inchingolo, R.; Torri, E.; Perrone, T.; Mento, F.; Demi, L.; et al. Integrating Domain Knowledge Into Deep Networks for Lung Ultrasound With Applications to COVID-19. IEEE Trans. Med. Imaging 2022, 41, 571–581. [Google Scholar] [CrossRef]
  61. Roshankhah, R.; Karbalaeisadegh, Y.; Greer, H.; Mento, F.; Soldati, G.; Smargiassi, A.; Inchingolo, R.; Torri, E.; Perrone, T.; Aylward, S.; et al. Investigating training-test data splitting strategies for automated segmentation and scoring of COVID-19 lung ultrasound images. J. Acoust. Soc. Am. 2021, 150, 4118. [Google Scholar] [CrossRef] [PubMed]
  62. Mento, F.; Perrone, T.; Fiengo, A.; Smargiassi, A.; Inchingolo, R.; Soldati, G.; Demi, L. Deep learning applied to lung ultrasound videos for scoring COVID-19 patients: A multicenter study. J. Acoust. Soc. Am. 2021, 149, 3626. [Google Scholar] [CrossRef] [PubMed]
  63. Wang, J.; Yang, X.; Zhou, B.; Sohn, J.J.; Zhou, J.; Jacob, J.T.; Higgins, K.A.; Bradley, J.D.; Liu, T. Review of Machine Learning in Lung Ultrasound in COVID-19 Pandemic. J. Imaging 2022, 8, 65. [Google Scholar] [CrossRef] [PubMed]
  64. Faucoz, O.; Standarovski, D.; Aguersif, A.; Bouharaoua, S.; Sarton, B.; Silva, S.; The MAGELLAN Study Group. Moving beyond the lines: Lung ultrasound pixel-wise computer-assisted analysis for critically ill patients. Crit. Care 2023, 27, 68. [Google Scholar] [CrossRef] [PubMed]
  65. Smit, M.R.; Hagens, L.A.; Heijnen, N.F.L.; Pisani, L.; Cherpanath, T.G.V.; Dongelmans, D.A.; de Grooth, H.S.; Pierrakos, C.; Tuinman, P.R.; Zimatore, C.; et al. Lung Ultrasound Prediction Model for Acute Respiratory Distress Syndrome: A Multicenter Prospective Observational Study. Am. J. Respir. Crit. Care Med. 2023, 207, 1591–1601. [Google Scholar] [CrossRef] [PubMed]
  66. Naveenkumar, E.; Dhiyanesh, B.; Kanna, R.R.; Diwakar, P.S.; Murali, M.; Radha, R. Detection of Lung Ultrasound COVID-19 Disease Patients based Convolution Multifacet Analytics using Deep Learning. In Proceedings of the 2022 Second International Conference on Artificial Intelligence and Smart Energy (ICAIS), Coimbatore, India, 23–25 February 2022; pp. 185–190. [Google Scholar] [CrossRef]
  67. Zhou, J.; Hu, B.; Feng, W.; Zhang, Z.; Fu, X.; Shao, H.; Wang, H.; Jin, L.; Ai, S.; Ji, Y. An ensemble deep learning model for risk stratification of invasive lung adenocarcinoma using thin-slice CT. NPJ Digit. Med. 2023, 6, 119. [Google Scholar] [CrossRef]
  68. Mohamed, T.I.A.; Oyelade, O.N.; Ezugwu, A.E. Automatic detection and classification of lung cancer CT scans based on deep learning and ebola optimization search algorithm. PLoS ONE 2023, 18, e0285796. [Google Scholar] [CrossRef]
  69. Sultan, L.R.; Chen, Y.T.; Cary, T.W.; Ashi, K.; Sehgal, C.M. Quantitative pleural line characterization outperforms traditional lung texture ultrasound features in detection of COVID-19. J. Am. Coll. Emerg. Phys. Open 2021, 2, e12418. [Google Scholar] [CrossRef]
  70. Sehgal, C.M.; Sultan, L.R.; Cary, T.W. Analysis of Pleural Lines for the Diagnosis of Lung Conditions. U.S. Patent Application No. 17/951,985, 23 March 2023. [Google Scholar]
  71. Roy, S.; Menapace, W.; Oei, S.; Luijten, B.; Fini, E.; Saltori, C.; Huijben, I.; Chennakeshava, N.; Mento, F.; Sentelli, A. Deep learning for classification and localization of COVID-19 markers in point-of-care lung ultrasound. IEEE Trans. Med. Imaging 2020, 39, 2676–2687. [Google Scholar] [CrossRef]
  72. Susanti, H.; Suprijanto. Image Processing Framework for Pleural Line (A-Line) Detection in Video Lung Ultrasonography. In Proceedings of the 2020 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), Langkawi Island, Malaysia, 1–3 March 2021; pp. 99–102. [Google Scholar] [CrossRef]
  73. Carrer, L.; Donini, E.; Marinelli, D.; Zanetti, M.; Mento, F.; Torri, E.; Smargiassi, A.; Inchingolo, R.; Soldati, G.; Demi, L.; et al. Automatic Pleural Line Extraction and COVID-19 Scoring From Lung Ultrasound Data. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 2020, 67, 2207–2217. [Google Scholar] [CrossRef]
  74. Hassan, A.E.; Desai, S.K.; Georgiadis, A.L.; Tekle, W.G. Augmented reality enhanced tele-proctoring platform to intraoperatively support a neuro-endovascular surgery fellow. Interv. Neuroradiol. 2021, 28, 277–282. [Google Scholar] [CrossRef] [PubMed]
  75. Ong, T.; Wilczewski, H.; Paige, S.R.; Soni, H.; Welch, B.M.; Bunnell, B.E. Extended Reality for Enhanced Telehealth During and Beyond COVID-19: Viewpoint. JMIR Serious Games 2021, 9, e26520. [Google Scholar] [CrossRef] [PubMed]
  76. Sparwasser, P.; Haack, M.; Frey, L.; Haferkamp, A.; Borgmann, H. Virtual und Augmented Reality in der Urologie [Virtual and augmented reality in urology]. Urologe A 2022, 61, 133–141. [Google Scholar] [CrossRef] [PubMed]
  77. Forte, M.P.; Gourishetti, R.; Javot, B.; Engler, T.; Gomez, E.D.; Kuchenbecker, K.J. Design of interactive augmented reality functions for robotic surgery and evaluation in dry-lab lymphadenectomy. Int. J. Med. Robot. 2021, 18, e2351. [Google Scholar] [CrossRef] [PubMed]
  78. Oulefki, A.; Agaian, S.; Trongtirakul, T.; Benbelkacem, S.; Aouam, D.; Zenati-Henda, N.; Abdelli, M.-L. Virtual Reality visualization for computerized COVID-19 lesion segmentation and interpretation. Biomed. Signal Process. Control 2022, 73, 103371. [Google Scholar] [CrossRef] [PubMed]
  79. Amara, K.; Aouf, A.; Kennouche, H.; Djekoune, A.O.; Zenati, N.; Kerdjidj, O.; Ferguene, F. COVIR: A virtual rendering of a novel NN architecture O-Net for COVID-19 Ct-scan automatic lung lesions segmentation. Comput. Graph. 2022, 104, 11–23. [Google Scholar] [CrossRef]
  80. Amara, K.; Kerdjidj, O.; Guerroudji, M.A.; Zenati, N.; Djekoune, O. Augmented Reality Visualization and Interaction for COVID-19 CT-Scan NN Automated Segmentation: A Validation Study. IEEE Sens. J. 2023, 23, 12114–12123. [Google Scholar] [CrossRef]
Ai 04 00044 g001
Figure 1. Schematic showing the computer-based image analysis steps. (A) demonstrates the semi-automated image segmentations steps. (B) shows the feature extraction and selection step towards building the diagnostic model.
Figure 1. Schematic showing the computer-based image analysis steps. (A) demonstrates the semi-automated image segmentations steps. (B) shows the feature extraction and selection step towards building the diagnostic model.
Ai 04 00044 g001
Ai 04 00044 g002
Figure 2. Demonstrates the high accuracy of semi-automated segmentation method of pleural lines. The left panels (a,b) show an example of a normal case, while panels (c,d) show a confirmed COVID-19 case, where p-lines in both cases were detected accurately by semi-automated segmentation.
Figure 2. Demonstrates the high accuracy of semi-automated segmentation method of pleural lines. The left panels (a,b) show an example of a normal case, while panels (c,d) show a confirmed COVID-19 case, where p-lines in both cases were detected accurately by semi-automated segmentation.
Ai 04 00044 g002
Table 1. The clinical and educational backgrounds of AI-analysis tool users.
Table 1. The clinical and educational backgrounds of AI-analysis tool users.
Years of Clinical ExperienceExperience in Diagnostic MethodsExperience with Ultrasound TechnologiesEducational Background
User 1None>30 years>30 yearsPhD
User 2>5 years>5 years>5 yearsMD
User 3>5 years>1 years>1 yearsMD
User 4None>20 years>20 yearsPhD
User 510 years10 years10 yearsMD
Table 2. Feature classification and diagnostic performance of quantitative ultrasound features by different users.
Table 2. Feature classification and diagnostic performance of quantitative ultrasound features by different users.
ThicknessThickness VariationPIDNonlinearityTortuosityEcho IntensityEcho HeterogeneityOverall Performance (AUC)
User 1COVID-195.200.232.680.251.54186.8618.45
Normal1.800.060.700.811.04195.0614.690.96
p-value0.040.040.010.010.010.470.12
User 2COVID-194.621.972.450.201.46184.7920.96
Normal1.410.360.490.901.01197.4118.600.98
p-value0.000.000.000.000.000.300.43
User 3COVID-196.030.252.450.231.33129.2034.46
Normal2.510.060.590.741.08137.7432.380.92
p-value0.030.000.000.000.070.200.53
User 4COVID-194.690.671.620.231.51202.2113.91
Normal1.100.851.840.791.16212.3111.500.84
p-value0.000.690.840.040.020.510.06
User 5COVID-195.841.403.030.131.10167.2134.62
Normal2.160.440.930.781.04182.4827.560.99
p-value0.000.010.010.000.040.220.04
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Sultan, L.R.; Haertter, A.; Al-Hasani, M.; Demiris, G.; Cary, T.W.; Tung-Chen, Y.; Sehgal, C.M. Can Artificial Intelligence Aid Diagnosis by Teleguided Point-of-Care Ultrasound? A Pilot Study for Evaluating a Novel Computer Algorithm for COVID-19 Diagnosis Using Lung Ultrasound. AI 2023, 4, 875-887. https://doi.org/10.3390/ai4040044

AMA Style

Sultan LR, Haertter A, Al-Hasani M, Demiris G, Cary TW, Tung-Chen Y, Sehgal CM. Can Artificial Intelligence Aid Diagnosis by Teleguided Point-of-Care Ultrasound? A Pilot Study for Evaluating a Novel Computer Algorithm for COVID-19 Diagnosis Using Lung Ultrasound. AI. 2023; 4(4):875-887. https://doi.org/10.3390/ai4040044

Chicago/Turabian Style

Sultan, Laith R., Allison Haertter, Maryam Al-Hasani, George Demiris, Theodore W. Cary, Yale Tung-Chen, and Chandra M. Sehgal. 2023. "Can Artificial Intelligence Aid Diagnosis by Teleguided Point-of-Care Ultrasound? A Pilot Study for Evaluating a Novel Computer Algorithm for COVID-19 Diagnosis Using Lung Ultrasound" AI 4, no. 4: 875-887. https://doi.org/10.3390/ai4040044

Article Metrics

Back to TopTop