Generative AI in Medicine and Healthcare: Promises, Opportunities and Challenges

Abstract

Generative AI (artificial intelligence) refers to algorithms and models, such as OpenAI’s ChatGPT, that can be prompted to generate various types of content. In this narrative review, we present a selection of representative examples of generative AI applications in medicine and healthcare. We then briefly discuss some associated issues, such as trust, veracity, clinical safety and reliability, privacy, copyrights, ownership, and opportunities, e.g., AI-driven conversational user interfaces for friendlier human-computer interaction. We conclude that generative AI will play an increasingly important role in medicine and healthcare as it further evolves and gets better tailored to the unique settings and requirements of the medical domain and as the laws, policies and regulatory frameworks surrounding its use start taking shape.

1. Introduction

Over the years, artificial intelligence (AI) has propelled revolutionary advancements across diverse industries, and its influence on healthcare can be particularly profound. Among the rapidly evolving AI technologies, generative AI models, such as the Generative Pre-trained Transformer (GPT) models developed by OpenAI with the popular ChatGPT model receiving the most attention, have emerged as powerful tools with the potential to reshape the landscape of healthcare due to their remarkable capacity for natural language processing (NLP) [1,2]. These advanced language models display an uncanny ability to comprehend and generate human-like text, making them ideal candidates for many applications, particularly medicine and healthcare. By leveraging vast amounts of medical data and knowledge, GPT models can transform various aspects of the healthcare industry, offering a new era of clinical decision support, patient communication, and data management. Their potential to process and interpret complex medical information has sparked optimism regarding their transformative impact on healthcare practices.

Through their applications in clinical decision support, GPT models can assist healthcare professionals in formulating their suggestions for optimizing their decision-making, leading to improved patient outcomes and the overall quality of healthcare services [3]. For instance, by analyzing vast medical datasets, GPT models can aid disease diagnosis and prognosis to identify and predict various medical conditions, facilitating earlier detection and personalized treatment strategies. Combined with extensive tooling, GPT models can assist radiologists in clinical diagnosis with the interpretation of medical images to enhance diagnostic accuracy and reduce interpretation time [4]. By leveraging their ability to comprehend complex molecular interactions, GPT models also can revolutionize drug discovery processes by predicting potential drug candidates with a higher likelihood of efficacy and safety, thereby accelerating the development of novel therapies and treatments [5].

In addition to enhancing the efficiency and quality of medical services, GPT models exhibit the potential to revolutionize patient communication. As interactive AI language models, GPT models can engage with patients, provide educational resources, and address medical queries, fostering better patient engagement and empowerment in managing their health [6]. Furthermore, the capacity of GPT models to streamline electronic health record (EHR) management and clinical documentation presents a potential avenue for alleviating administrative burdens, allowing medical practitioners to focus more on patient care [7].

Despite the transformative potential of generative models, their integration in medicine and healthcare is not without challenges and ethical considerations. Ensuring the accuracy and reliability of AI-driven decisions remains a critical concern, particularly in critical medical contexts. The “black box” nature of some AI models, including generative models, raises questions about the interpretability of the decisions they generate, calling for greater transparency and explainability in AI systems utilized in healthcare. Moreover, ethical considerations concerning data privacy, patient confidentiality, and potential biases in AI models require careful attention [8]. As these models interact with sensitive medical information, patient privacy and data security are paramount to maintaining public trust in AI-enabled healthcare solutions.

In light of recent developments, this review article provides an overview of existing industry and research efforts of applying generative AI models in medicine and healthcare. By highlighting their vast potential, advantages, challenges, and ethical considerations, this study seeks to contribute to the ongoing dialogue on harnessing AI’s transformative capabilities responsibly for the betterment of medical practice and patient well-being. As we explore generative models’ impact on healthcare and medicine, navigating the ever-evolving AI landscape with a commitment to ethical principles, patient-centred care, and a collaborative approach between AI developers, medical professionals, and policymakers is crucial.

2. Background

NLP is a pivotal subfield of artificial intelligence (AI) that focuses on the interaction between computers and human language. Its primary objective is to enable machines to comprehend, interpret, and generate human language in a meaningful and contextually relevant manner. NLP encompasses a wide spectrum of tasks, including language translation, sentiment analysis, speech recognition, text summarization, question answering, and more, each designed to bridge the gap between human communication and computational understanding [9].

NLP has witnessed extensive implementation in healthcare, showcasing its remarkable ability to extract and analyze valuable insights from vast amounts of unstructured clinical data, including EHRs, medical literature, and patient-generated content [10]. Notably, NLP has shown significant promise in converting unstructured clinical notes into structured data, helping to identify medical conditions, medications, and lab test names through tasks like named entity recognition [11,12]. Similarly, NLP has been applied to unstructured EHR data to identify and detect adverse drug events using identified drugs and their interactions [13]. NLP-driven models have also contributed to early disease detection, enabling timely interventions and improving patient outcomes [14].

Within the realm of NLP, the GPT models have emerged as a noteworthy development pioneered by OpenAI. GPT models constitute a family of sophisticated language models designed to harness the potential of deep-learning neural networks. The transformer architecture, first proposed by Vaswani et al. in 2017 [15], underlies the functioning of GPT models. This neural network architecture excels in processing sequential data, especially text, by employing “self-attention”. Self-attention enables the model to understand the importance of and relationships between individual words in a sentence during processing, allowing it to give higher attention to semantically related words and carry more contextual information. The revolutionary aspect of transformers lies in their capability to handle long-range dependencies within language sequences efficiently, enhancing their proficiency in contextual language understanding and semantic comprehension.

Since their reception, the development and evolution of GPT models have witnessed rapid progress over a relatively short period. The timeline of GPT models begins with the introduction of the original GPT-1 in June 2018 by OpenAI. GPT-1 demonstrated the potential of pre-training large-scale Transformer models on vast amounts of text data to generate coherent and contextually relevant language. Building upon the success of GPT-1, OpenAI released GPT-2 in February 2019. GPT-2 was a significant advancement, boasting an impressive 1.5 billion parameters, but its release raised concerns about potential misuse due to its ability to generate human-like text, including fake news and misinformation. As a result, OpenAI initially limited the release of GPT-2, only providing access to smaller versions of the model. In November 2019, OpenAI fully released the GPT-2 model, making it widely accessible to researchers and developers. This move led to an explosion of research and experimentation, driving the rapid development and fine-tuning of GPT-2 across various domains, including text generation, chatbots, language translation, and more [1,16].

The timeline took another leap in June 2020 with the release of GPT-3. This model was a monumental achievement, consisting of a staggering 175 billion parameters, making it the largest language model ever created at that time. GPT-3 demonstrated unparalleled language generation capabilities, including the ability to perform tasks like translation, summarization, question-answering, and even creative writing. GPT-3’s success ignited further interest in LLMs, leading to numerous research breakthroughs and commercial applications [1]. In November 2022, OpenAI released yet another revolutionary ChatGPT model as an advanced conversational AI powered by the GPT-3.5 architecture. It is designed to engage in natural and dynamic conversations, making it a versatile and useful tool for various applications. Users can engage with ChatGPT by providing text prompts or questions. In response, ChatGPT generates coherent and contextually relevant answers based on its acquired knowledge from the pre-training data. Five months later, GPT-4 was introduced in March 2023 as a large multimodal model that accepts image and textual inputs, which has even more improved performance over ChatGPT. Given the large scale of the pre-training step, these trained models are also known as Large Language Models (LLMs) [16].

These models’ inherent “generative” nature grants them the extraordinary ability to produce human-like text autonomously, which has significant implications for a wide range of domains, including healthcare. Pre-training is a crucial step in creating GPT models. During this phase, the model is exposed to a massive corpus of diverse text and datasets, enabling it to acquire a robust understanding of language patterns and structures. The fusion of pre-training and transformer architecture equips GPT models with the capability to excel in various NLP tasks, facilitating text generation with high fluency and contextual coherence similar to humans [17].

Pre-training is akin to the initial phase of knowledge acquisition, where GPT learns from vast amounts of unlabeled text data to develop a strong language understanding. Subsequently, fine-tuning, the next phase of GPT model training, takes place using task-specific, labeled data [17], such as data in healthcare. This process adapts the pre-trained model to perform specific NLP tasks relevant to healthcare, such as medical question answering, clinical text classification, or language translation of medical records.

The confluence of GPT’s language understanding capabilities, coupled with pre-training and fine-tuning, allows the model’s acquired medical language knowledge to excel in a wide range of downstream healthcare NLP tasks. This leads to enhanced model performance and reduces the time and resources required for comprehensive model training. By generating human-like text and leveraging their language knowledge in healthcare-specific tasks, LLMs have demonstrated tremendous potential in advancing AI-driven healthcare solutions. The integration of generative AI in healthcare holds promise for enhancing clinical decision support, improving patient communication, and accelerating disease diagnosis, ultimately opening up new avenues for AI-driven healthcare solutions that can assist medical professionals in making well-informed decisions and improving patient care [7].

3. Current Efforts of Applying Generative AI and LLMs in Medicine and Healthcare

This section reviews the current efforts and research initiatives that aim to apply Generative AI and LLMs to enhance various aspects of medical practice, ranging from providing clinical administration support to providing resources for professional education for both providers and patients. The integration of these cutting-edge technologies holds considerable promise in augmenting patient care, facilitating medical research, and alleviating the workload on healthcare professionals. By investigating the latest advancements in this field, we seek to gain insights into the transformative potential of state-of-art AI language models in shaping the future landscape of healthcare.

3.1. Clinical Administration Support

One prominent application of generative AI models in healthcare is the automation of clinical documentation that provides clinical administration support. Busy clinicians, often burdened with extensive note-taking, can leverage ChatGPT’s capabilities to generate draft clinical notes swiftly and accurately. By providing a brief verbal summary (a “prompt”) or relevant patient data (given data privacy is respected), comprehensive and contextually relevant clinical documentation can be generated to save clinicians’ time. Microsoft Copilot [18] is an enterprise tool that integrates generative AI into everyday tools like Word, PowerPoint, Teams, and others to improve productivity. This integration has a powerful potential to facilitate multidisciplinary collaboration among healthcare teams. For example, when working with complex cases involving multiple specialities, a generative AI-based meeting tool can assist in creating meeting agendas, identifying suitable team members for follow-up actions, and summarizing key points from meetings.

AI-powered healthcare solutions offered by Nuance enhance the efficiency and effectiveness of healthcare professionals in various clinical settings. Nuance’s speech recognition technology plays a significant role in clinical documentation improvement, allowing clinicians to dictate their notes directly into the EHR system. This not only saves time but also enhances the accuracy of patient information capture. For example, the tool could transcribe a hematologist’s verbally reported findings as they examine blood smears in real time [19]. Similarly, Suki Assistant [20] automates clinical note creation by listening to clinician-patient interactions, reducing administrative burdens. It offers flexible interaction options, such as dictation or ambient note generation, and provides diagnosis code suggestions. For example, a hematologist can use Suki to automatically capture and create clinical notes during a consultation for chronic lymphocytic leukemia, which could allow more face time with the patient. This streamlines documentation tasks, allowing more time for patient care and addressing physician burnout. Suki Assistant’s applications exemplify the potential of generative AI in enhancing clinical workflows and improving healthcare professionals’ efficiency and well-being.

Corti [21] is another tool that utilizes generative AI to offer real-time transcription, guidance, and coding capabilities across a variety of communication channels. Through automated transcription of patient dialogues in real-time and multiple languages, Corti ensures the preservation of critical information, minimizing the risk of manual transcription errors or delays. Additionally, Corti’s capability to extract vital details from transcribed dialogues, including specific symptoms, mentioned medications, and critical questions, streamlines the review of essential encounter highlights. Leveraging the extracted information, Corti’s AI provides recommendations for optimal patient care, leveraging a vast database of diverse data points to determine the most suitable next steps. Moreover, following the patient encounter, Corti assists in documenting the interaction by automatically coding procedure and diagnosis codes, such as ICD-10 and CPT, leading to time savings and minimizing the potential for human error. This ensures the accuracy of patient records and facilitates efficient billing procedures [21].

Google Bard [22], powered by Med-PaLM 2, offers exciting applications in healthcare, especially in providing 24/7 patient support and assisting clinicians. Trained in diverse medical information, including journals, textbooks, clinical notes, and patient records, Med-PaLM 2 enhances Google Bard’s capabilities in generating medical content. The tool can aid in answering patient queries, suggesting possible diagnoses, and supporting treatment plans. In hematology, it can provide information and support to patients with blood disorders, offering immediate responses and recommending professional medical attention if needed. However, it is essential to use AI-generated responses for informational purposes only and not as a substitute for professional medical advice. As Google Bard continues to develop, its potential to revolutionize healthcare interactions and improve patient care remains promising.

Ellen AI [23], an algorithm complementing generative AI tools like ChatGPT, has valuable applications in healthcare. By providing a text-to-voice interaction layer, Ellen AI offers auditory explanations to support patient care. Healthcare clinicians can leverage their capabilities to enhance patient communication and accessibility by converting written instructions into high-quality spoken content. Meanwhile, ChatGPT’s extensive generative AI properties have potential in clinical and administrative tasks, including data analysis, decision support, and treatment plan comprehension and adherence. The combination of Ellen AI and ChatGPT holds promising opportunities to improve patient care and healthcare efficiency through innovative voice-based interactions and intelligent text generation [24].

3.2. Clinical Decision Support

Given the advanced understanding of the human language and further fine-tuned domain knowledge, GPT models also have the potential to support clinical decision-making. Glass AI [25] is an LLM-powered experimental tool that offers clinical decision support. It serves as a diagnostic assistant to generate a list of possible diagnoses and treatment plans tailored to a clinical audience. For instance, when presented with a patient exhibiting symptoms such as fatigue, shortness of breath, and paleness, a provider can input these symptoms into Glass AI [26]. The system can then produce a comprehensive differential diagnosis, potentially suggesting conditions such as anemia, leukemia, or myelodysplastic syndromes. Additionally, Glass AI can assist in formulating a clinical plan, guiding the hematologist’s next steps for further tests or treatment.

Regard [27] is an AI tool integrated with EHR systems. By analyzing patient data, it suggests diagnoses, writes clinical notes, and provides relevant information quickly, optimizing patient care. It automates some EHR-related administrative tasks, enabling clinicians to focus more on patients and less on searching through screens. Regard’s generative AI capabilities assist in the diagnostic process, generating a list of plausible and novel differential diagnoses based on patient data. This aids healthcare professionals (HCPs) in exploring different possibilities, confirming, or ruling out diagnoses, and optimizing treatment plans. For primary care physicians, hematologist-oncologists, dermatologists, and other specialists, Regard provides evidence-based suggestions, saving time and improving diagnostic specificity. As an intelligent co-pilot, Regard enhances HCPs’ use of EHRs, promoting faster and more informed decision-making while emphasizing that it supports, rather than replaces, their clinical judgment and expertise. In pilot programs, Regard has demonstrated significant time-saving benefits and improved diagnostic accuracy for physicians.

Redbrick AI’s Fast Automated Segmentation Tool (F.A.S.T) [28] offers significant applications in medical imaging, assisting healthcare professionals in annotating and segmenting CT scans, MRI images, and ultrasounds. Redbrick AI provides a SaaS platform for annotating medical image data and offers F.A.S.T for use in radiology by utilizing Meta’s Segment Anything model [29], presenting a possible solution for enhancing diagnostic accuracy and speed in healthcare. Its adaptive nature simplifies accurate segmentation without additional data, making it valuable for segmenting visible objects and features in radiology. The tool’s real-time interaction allows clinicians to witness mask computation, streamlining segmentation. F.A.S.T. automates manual segmentation, enhancing diagnostic accuracy and speed in radiology.

Paige FullFocus [30], fueled by generative AI, empowers HCPs to view, manage, and share digital slides of tissue samples, providing novel insights for treatment decisions and improving accuracy, efficiency, and diagnostic confidence. It excels in identifying and analyzing complex tissue patterns, aiding precise diagnoses in challenging cases like counting cancer cells in prostate and breast biopsies and identifying biomarkers for treatment selection. Additionally, FullFocus supports HCPs’ clinical practice and education, enabling them to study diverse tissue patterns, expand their knowledge of hematological and oncological conditions, and stay updated with the latest pathology advancements. With FullFocus, HCPs can refine their diagnostic skills, deepen their expertise, and enhance patient care through continuous learning. The promising capability of Paige tools to provide more accurate tumor diagnosis was demonstrated in a prostate cancer research study conducted by Raciti et al. [31].

Kahun [32] is a symptom checker tool empowered by a conversational chatbot integrated with the EHR. The tool provides clinical assessments of patients by producing ranked differential diagnoses and workup options based on patient input and medical knowledge. Kahun’s AI inference engine delivers a ranked list of potential diagnoses, speeding up the diagnostic process and saving time. Ben-Shabat et al. demonstrated Kahun’s superior performance compared to a selected set of similar AI symptom checkers [33]. Further workup options are suggested for comprehensive patient evaluation. Kahun’s growing network of relationships between disorders, complications, and findings keeps healthcare professionals updated with the latest medical knowledge.

3.3. Patient Engagement

Hippocratic AI [34] focuses on creating an LLM tailored for healthcare. It aims to offer one that is patient-centred, prioritising empathy, care, compassion, and generation of patient-friendly responses, enhancing patient engagement and outreach. This important notion of ‘generative AI empathy’ has been demonstrated in a study by Ayers et al., who reported that LLM-powered chatbot (ChatGPT) responses were preferred over physician responses and rated significantly higher for empathy [35].

By focusing on non-diagnostic, patient-facing applications, Hippocratic AI values patient safety while improving healthcare access and outcomes. Hippocratic AI proves beneficial in augmenting administrative tasks and handling complexities like medical coding and licensure exams. Moreover, its compliance certifications demonstrate reliability in maintaining healthcare standards. In clinical settings, the model’s exceptional performance on various medical certification exams confirms its real-world applicability. By providing accurate and empathetic support to healthcare professionals, Hippocratic AI enriches patient care, fostering a trustful and efficient healthcare environment.

Gridspace [36] is an enterprise solution powered by generative AI that automates patient outreach by handling phone calls, answering questions, and performing administrative tasks. It enables scalable, 24/7 accessible, cost-effective patient engagement, receiving inbound calls and phoning patients. Gridspace automates routine administrative tasks such as appointment scheduling, patient reminders, insurance verification, and more. By offloading these inquiries to voice bots, healthcare professionals can save time and focus on critical patient care tasks. Furthermore, Gridspace can triage and direct patient inquiries in real-time, providing valuable insights. Its applications exemplify the potential of Generative AI in transforming patient interactions, streamlining administrative processes, and enhancing overall healthcare efficiency and patient satisfaction.

3.4. Synthetic Data Generation

Syntegra Medical Mind [37] utilizes generative AI to produce realistic synthetic patient records from real healthcare data like EHRs while protecting patient privacy. Healthcare professionals can access and analyze this data for research, education, and decision-making without compromising confidentiality. The synthetic records match the statistical properties of the original data, including rare cohorts and outliers, aiding specialists in understanding diverse disease patterns. Syntegra also addresses data bias and improves algorithmic fairness, promoting equitable treatment plans. The synthetic data layer breaks barriers to data access, fostering innovation and enhancing patient care. Muniz-Terrera et al. [38] demonstrated the potential to advance dementia research through virtual cohorts synthesized with Syntegra.

DALL-E 2 [39] is another OpenAI model for text-to-image generation. It was trained on billions of text-image pairs to learn to create realistic synthetic images. Thanks to its extensive pretraining, DALL-E 2 has the exciting potential of creating or augmenting medical data that is often sparse or limited in medical research and education without compromising patient privacy. Adams et al. [40] investigated the DALL-E 2′s domain knowledge in radiology by testing the output medical images generated from short descriptions and those from reconstructing existing radiological images with missing areas. The study showed that even though generation performance suffered for complex images such as CT, MRI, and ultrasound, DALL-E 2 could produce x-ray images that maintained similar style and anatomical proportions to authentic images, given that they were without pathologies. Despite the limited capabilities of directly applying DALL-E 2 in medical image generation, it showed promising potential to be further fine-tuned with medical data and related terminology to create a customized model for radiological data generation and augmentation.

3.5. Professional Education

A UNESCO quick start guide published in April 2023 describes a number of potential use cases of generative AI, such as ChatGPT, in higher education, including application examples in teaching and learning and academic research [41]. Although focused on higher education in general with no specific mention of medical education, many of these applications can be adapted to the specific settings of medical education at undergraduate, postgraduate and continuing medical education (CME) levels (and even to patient education and health education of the general public).

The second author of this article was recently guest editing a theme issue for JMIR Medical Education on the topic of ‘ChatGPT and Generative Language Models in Medical Education’. The issue attracted dozens of submissions in a few months, with more than a dozen of these already published online as of 31 July 2023, covering a range of generative AI topics in medical and professional education [42].

Unlearn.AI [43] utilizes generative AI to create “digital twins” of individual patients, offering a comprehensive model of potential health outcomes under different scenarios. Healthcare professionals can personalize treatment plans, monitor patient progress, and make informed decisions for improved patient outcomes. The digital twins simulate the effects of various treatments on a patient’s biology using real-world data, aiding in personalized treatment plans and clinical research. In hematology, for instance, clinicians can use digital twins to simulate disease progression under different therapies, guiding treatment choices. Additionally, digital twins optimize clinical trials, providing insights without large control groups. Unlearn. AI’s approach streamlines processes, supporting clinician education and enhancing patient follow-up. The study by Bertolini et al. also demonstrated accuracy in capturing disease progression, making digital twins a valuable tool in advancing healthcare practices [44].

As patients increasingly record their doctor’s visits, research has been underway to use these transcripts to gain insights for patients and extract structured data for enriching EHRs. Abridge [45] is a digital tool that utilizes generative AI to document medical dialogues, saving physicians from manual note-taking. Abridge plays a vital role in patient education by sending after-visit summaries to patients through its consumer app, aiming to increase their engagement and adherence to care. In cases like polycythemia vera, patients may struggle to remember all the details discussed during their appointments. Abridge tackles this problem by providing a comprehensive transcript of the conversation for patients to review later. The platform also highlights essential information from the dialogue and simplifies complex medical jargon into more understandable language. This ensures that patients have a clear comprehension of their diagnosis, treatment choices, and future actions, ultimately promoting better patient adherence and positive health outcomes.

Additionally, Krishna et al. presented end-to-end methods for generating long and semi-structured clinical summaries called SOAP notes from clinical conversations in collaboration with Abridge [46]. Their approach leveraged a unique corpus of transcripts and associated SOAP notes. The methods focus on decomposing summarization tasks into extractive and abstractive subtasks, shifting the workload progressively from the abstractive to the extractive component.

3.6. Examples from Europe and Asia

The generative AI tool examples described above (under 3.1 to 3.5) were mostly developed in the USA, but Europe and Asia are rapidly catching up. For example, the Dutch medical technology giant Philips is currently (at the time of writing) developing generative AI applications to enhance its PACS (Picture Archiving and Communication System) image processing and diagnostic capabilities and simplify clinical workflows [47], whilst in Asia, SayHeart, a startup based in Malaysia and Singapore, has launched a new algorithm in August 2023 that can seamlessly translate medical jargon, health reports and complex imaging into visual, easily accessible content [48]. Moreover, Riken, a large scientific research institute in Japan founded in 1917, has embarked on an eight-year generative AI research programme (2023–2031) to generate medical and scientific hypotheses by learning from research papers and images [49].

4. Discussion

Generative AI is poised to revolutionize medicine during the coming years [50]. The application examples presented in this article are a glimpse of what will come. A quick PubMed query using the term ‘ChatGPT’ retrieved 924 publication records as of 31 July 2023 (4 records in 2022 and 920 records in the first seven months of 2023; the same query retrieved 1049 records when repeated 18 days later, 18 August 2023) [51]. Whilst ChatGPT has no doubt dominated the generative AI scene in 2023, it should be noted that it is but one example of the GPT architecture and models. We expect generative AI to continue trending during the coming months and years. The following discussion will address some common concerns, challenges and general opportunities associated with generative AI and related products such as ChatGPT.

4.1. Can We Trust Generative AI? Is It Clinically Safe and Reliable?

Trust and validation are essential to generative AI’s adoption success in medicine and healthcare. ChatGPT’s responses have shown a wide, and most importantly unpredictable, fluctuation in quality and veracity. This ‘unpredictability’ is the main barrier to adoption success, as we do not know when it is going to return a good answer and when its answers are going to be wrong or misleading, or in other words, when to trust generative AI and when not to trust it, especially when the user is not sufficiently qualified to assess the quality (accuracy and completeness) of a given response. ChatGPT (at the time of writing), for example, is known to make stuff up by inventing and citing academic papers that do not exist [52,53,54]. This phenomenon, also known as generative AI “hallucinations”, can be reduced using techniques such as Retrieval Augmented Generation (RAG) [55]. Generative AI is also prone to various forms of bias depending on how it has been trained [41] and may not always perform equally well across different languages [56].

This discussion of trust also brings in the related issues of clinical safety and reliability. Until we have a properly medically trained and validated generative AI (ChatGPT, for example, is not specifically medically trained), there will always be these interrelated issues of trust, safety and reliability hindering any serious medical use of it. By medically trained, we mean a model that has been specifically and comprehensively trained using a corpus of quality evidence-based medical texts that sufficiently cover a given medical area of specialism.

4.2. Clinical Evaluation, Regulation and Certification Challenges

The problem is compounded by the ever-evolving nature of medical/clinical knowledge, which requires a matching form of generative AI that can be continually and reliably trained and updated. Furthermore, the rapid evolution of large language models and generative AI challenges their clinical evaluation, regulation, and certification.

For example, we already have multiple versions of OpenAI’s ChatGPT, e.g., GPT3.5, GPT-4 and DALL-E 2 (images) [57], and offerings from Meta (Llama 1 and 2, in partnership with Microsoft [58]) and Google (Bard [59], which still cannot provide answers to specific clinical cases at the time of writing, likely a Google-imposed artificial limitation). The more recent a model’s version is, the better it tends to perform, e.g., [60], but this is not always guaranteed [61]. We expect that dedicated, medically trained large language models and generative AI will similarly have multiple successive versions over short periods when introduced in the near future.

However, clinical evaluation and certification are processes that traditionally take a relatively long time to complete, so there is always this risk that by the time an evaluation is completed, the evaluated AI has already changed substantially with the release of a new version requiring a fresh evaluation. Regulatory bodies are trying to keep pace by implementing the necessary mechanisms for dealing with AI as a medical device [62]. However, LLMs bring new challenges compared with already regulated AI-based technologies and will therefore require additional regulatory adaptations [63].

4.3. Privacy Concerns

In April 2023, Italy blocked access to ChatGPT in the country due to privacy concerns, including concerns about its collection and storage of personal user data to further train its model [64]. A few weeks later, access was restored in Italy [65] following the introduction of new functionality in ChatGPT that allowed users to turn off chat history and thus choose which conversations could or could not be used to train the underlying models [66]. Nevertheless (and until healthcare organizations can afford [67] and run their own fully locally hosted and managed instances of these models and tools that can be trusted not to send any information to external companies or providers for processing), it is still highly recommended (if not mandatory in the case of confidential patient information, for example) that users never put any sensitive information or personal data into these tools [68]. (On the flip side, medical educators might find text-to-image generative AI useful in producing photo-quality teaching images depicting various clinical conditions without the confidentiality and consent concerns associated with using real patient photos, especially when whole-face images are necessary).

There is still no openness or transparency about these models’ training data or the code used to train them [69]. The unauthorized access (without consent) to data sources, including possibly private and confidential data sources, for generative AI learning and training is currently (at the time of writing) the subject of a court case filed in the USA [70]. Some researchers are already recommending that AI models follow privacy laws, including the ‘right to be forgotten’ and the ability to forget or unlearn what they have learned about situations or specific persons [71].

4.4. Copyright and Ownership Issues

The above-mentioned ‘access without consent’ lawsuit brings to attention potential copyright issues related to the data used to train these models [72]. There are also further unsettled copyright and intellectual property ownership questions regarding content generated by these models, e.g., new radiology images generated by DALL-E 2 in response to user’s text prompts [40]. Who owns the copyright to AI-generated content? Who should be held liable for any harm or loss it might cause? These issues become more complex when AI-generated content is based on copyrighted materials [73,74]. Interestingly, Microsoft introduced a new section on AI services to its overall Services agreement effective September 30, 2023, in which it expanded the definition of “Your Content” to include “content that is generated by your use of our AI services” [75].

4.5. Solutions on the Horizon

As with other emerging and rapidly developing technologies, the corresponding governing laws and regulations often lag and need some time to catch up. Some generative AI issues related to trust, safety, reliability, privacy, copyrights, and ownership are not yet fully settled (no definitive answers or solutions at the time of writing), but this does not mean they are unsurmountable. They will gradually get addressed over time as the technology evolves and matures and the laws, policies and regulatory frameworks surrounding its use start taking shape.

The use of AI, including generative AI, in the European Union (EU) is about to be regulated by a new AI Act [76], the world’s first comprehensive AI law expected to enter into force around mid-2024 with a grace period of a further 24–36 months before its main requirements become binding. The EU AI Act introduces new transparency requirements for generative AI, including a need to publish summaries of copyrighted data used for training. At the same time, medical and healthcare regulatory bodies, such as the Medicines and Healthcare Products Regulatory Agency (MHRA) in the UK, are introducing new regulatory frameworks and/or updating existing ones to specifically deal with AI as a medical device [62]. However, medical AI algorithms will often continue to learn from new data after initial regulatory approval and, therefore, may require ongoing reapproval at regular intervals, a process that has yet to be determined and implemented by regulators.

4.6. Opportunities for Custom Solutions and Much Improved User Interfaces

Third-party developers are now able to easily build their custom applications and solutions using APIs (Application Programming Interface) and plugins offered by leading generative AI providers, such as OpenAI [77,78,79] and Google [80]. For example, GPT-trainer, a SaaS (Software-as-a-Service) from Petal/Paladin Max, Inc., allows users to build and deploy their own ChatGPT assistants trained with their data without coding [81].

AI-driven, natural-language-based conversational (ChatGPT-like) user interfaces (UIs) are set to become the next big thing in the evolution of user experience design, much like the way mouse/point-and-click, speech and touch-based interfaces have revolutionized our interactions with computers and mobile devices [82]. Instead of asking users to adapt to and master rather rigid (pre-programmed) and less forgiving interfaces, UIs will more flexibly adapt to users’ needs more naturally. Users will simply be asked to describe in their way and words what they need done (Figure 1). GPT-OSM applies this concept to OpenStreetMap, allowing users to intuitively discover features on OpenStreetMap using natural language queries without the need to learn complex query languages or syntax [83]. Many healthcare UIs would benefit from the same approach, e.g., query UIs of electronic patient records and their aggregates and improved UIs for querying health digital twins (HDTs) [84]. In the case of HDTs, a ChatGPT-like UI can help bridge the gap between complex HDT data/models and their human users’ understanding and needs (users include healthy individuals, patients and clinicians).

Furthermore, by making coding easier or almost automated, Generative AI can speed up the development of some health applications. For example, ChatGPT was able to successfully code a remake of Beat Saber, a well-known virtual reality exergame [85]. Tao and Xu demonstrated the creation of thematic maps in ChatGPT using given or public geospatial data. ChatGPT successfully produced all the code needed to generate the maps [86] (cf. the above-mentioned OSM-GPT project).

Generative AI also holds promise for the Internet of Medical Things (IoMT [87]). For example, it can assist in generating new designs for edge-based medical and health monitoring devices [88]. Some of the associated software development (coding) tasks can additionally run some of these devices by continually learning user (e.g., patient or clinician) preferences, adapting to them, and providing better UIs and overall user experience [89,90]. Furthermore, it can generate synthetic and augmented data to test and improve the accuracy of machine learning algorithms (powering many smart medical devices) without requiring access to real patient data [90] or when the latter is not sufficient [91]. IoMT security is another area where generative AI can potentially help; for example, by auto-generating suitable mitigation activities in response to various threats [92].

5. Conclusions

In this article, we reviewed a selection of representative examples of generative AI applications in medicine and healthcare. We then briefly discussed some associated issues and concerns, such as trust, veracity, clinical safety and reliability, privacy, copyrights, and ownership, and the opportunity to use the technology to create friendlier AI-driven conversational user interfaces for health and healthcare applications. We expect that all these discussed concerns will gradually get addressed with time as the laws, policies and regulatory frameworks surrounding the use of generative AI begin taking shape. We share the belief of Lee, Goldberg and Kohane [47] that generative AI will play an increasingly important role in medicine and healthcare as it further evolves and gets better tailored to the unique settings and requirements of the medical domain. The coming years will see the introduction of new models specifically and comprehensively trained using corpora of quality evidence-based medical texts that sufficiently cover various clinical areas of specialism. These models will be of great help to healthcare professionals and their patients in the not-very-distant future. Rather than AI replacing humans (clinicians), we see it as “clinicians using AI” replacing “clinicians who do not use AI” in the following years [93].