A roadmap for designing more inclusive health chatbots

Chatbots is a software responsible for establishment of conversation between a human and artificial intelligence. These chatbots are either cloud-based or on premise solutions, which are used by patients for checking symptoms, locating clinics or scheduling appointments. Furthermore, healthcare chatbots are also used by healthcare payers to establish a relation between the company and the potential customers. There are various competitive “open software” platforms to ChatGPT that have been launched, including Google Bard among others (Schechner, 2023). Unfortunately, this raises the concern that ChatGPT and these open software platforms can invade the privacy of user content. It is quintessential to ensure quality care and reserved rights of the patients globally.

Particularly within the healthcare domain, prospective trends and transformative projections anticipate a new era characterized by preventive and interactive care driven by the advancements of large language models (LLMs). Second, by identifying the rational and irrational factors influencing individuals’ resistance to health chatbots, this study advances the established literature’s comprehension of resistance behavior toward health chatbots. Population health management increasingly uses predictive analytics to identify and guide health initiatives. In data analytics, predictive analytics is a discipline that significantly utilizes modeling, data mining, AI, and ML.

The National Health Service (NHS) has tested this app in north London, and now about 1.2 million people are using this AI chatbot to answer their questions instead of calling the NHS non-emergency number [85]. In addition, introducing intelligent speakers into the market has a significant benefit in the lives of elderly and chronically ill patients who are unable to use smartphone apps efficiently [86]. Overall, virtual health assistants have the potential to significantly improve the quality, efficiency, and cost of healthcare delivery while also increasing patient engagement and providing a better experience for them. With all the advances in medicine, effective disease diagnosis is still considered a challenge on a global scale. The development of early diagnostic tools is an ongoing challenge due to the complexity of the various disease mechanisms and the underlying symptoms. ML is an area of AI that uses data as an input resource in which the accuracy is highly dependent on the quantity as well as the quality of the input data that can combat some of the challenges and complexity of diagnosis [9].

AI chatbots are great for a quick response, but they have limitations in understanding context, and the information is not always accurate or up-to-date. You should fact-check AI chatbot responses before believing the information is true or following advice. A 2024 study from DUOS, a digital health company that offers an AI-powered service, found that 60% of respondents are willing to use AI tools to quickly answer health questions for themselves and older family members. The article also talked about how developers can design LLM-based tools that could be approved as medical devices, and creation of new frameworks that preserve patient safety. Writing in an article, Prof. Gilbert stated these chatbots are unsafe tools and stressed the need to develop new frameworks that ensure patient safety.

The algorithms rely on other AI approaches, such as machine learning, deep learning and natural language processing (NLP), to perform these tasks. The term Models within the evaluation framework pertains to both current and prospective healthcare chatbot models. The framework should enable seamless interaction with these models to facilitate efficient evaluation.

Artificial intelligence chatbots for the nutrition management of diabetes and the metabolic syndrome

Recent research published in JAMA Psychiatry demonstrated just how valuable these tools can be by detailing the development of an ML-based predictive model capable of accurately stratifying suicide risk among patients scheduled for an intake visit to outpatient mental healthcare. At their core, clinical decision support (CDS) systems are critical ChatGPT App tools designed to improve care quality and patient safety. But as technologies like AI and machine learning (ML) advance, they are transforming the clinical decision-making process. Their ability to improve accessibility and automation, and empower patients has made them invaluable tools in delivering efficient and patient-centered care.

Chatbot interventions were effective across a range of populations and age groups, with shorter- and longer-term, text and voice-based chatbots, chatbot-only and multicomponent interventions being effective. However, future large-scale trials, with rigorous study designs and outcome measures, and long-term follow-up are required to confirm these findings. Medical chatbots are especially useful since they can answer questions that definitely should not be ignored, questions asked by anxious patients or their caregivers, but which do not need highly trained medical professionals to answer.

Real-world success stories, such as the NHS 111 Online and Babylon Health’s GP at Hand, demonstrate the tangible impact of these technologies in streamlining patient triage and delivering personalized care at scale. Healthcare organizations, technology providers, and policymakers will need to collaborate to address these challenges and ensure that the benefits of AI-powered chatbots are realized while prioritizing patient safety and well-being. They collect patients’ symptoms and histories, manage test results, facilitate communication between doctors ChatGPT and patients and more. Implementing features that help secure medical data and access to it helps you to increase patients’ trust in your chatbot and comply with HIPAA, PIPEDA, GDPR and other data protection laws and regulations. My company also faces challenges related to the accuracy and security of AI chatbots in our attempts to make them reliable tools for healthcare organizations. Today, I want to share our experience and assure you that those challenges should not restrain you from leveraging generative AI for the benefit of your business.

Thought Experiments Based on Real Studies

AI tools can leverage large datasets and identify patterns to surpass human performance in several healthcare aspects. AI offers increased accuracy, reduced costs, and time savings while minimizing human errors. It can revolutionize personalized medicine, optimize medication dosages, enhance population health management, establish guidelines, provide virtual health assistants, support mental health care, improve patient education, and influence patient-physician trust. In response, the need for timely vaccination communication called for more effective use of social media and digital technologies such as machine learning, artificial intelligence, and conversation technology26. Among different digital interventions, chatbots have become an increasingly popular tool in health communication and services due to their ubiquitous access points and potential for massive information dissemination. However, as the use of chatbots in the context of health communication, especially in vaccine communication, is a novel approach, rigorous evaluations of their impact and potential use cases are very limited28.

AI algorithms can analyze large amounts of data and identify patterns and relationships that may not be obvious to human analysts; this can help improve the accuracy of predictive models and ensure that patients receive the most appropriate interventions. AI can also automate specific public health management tasks, such as patient outreach and care coordination [61, 62]. Which can help reduce healthcare costs and improve patient outcomes by ensuring patients receive timely and appropriate care. However, it is pivotal to note that the success of predictive analytics in public health management depends on the quality of data and the technological infrastructure used to develop and implement predictive models. In addition, human supervision is vital to ensure the appropriateness and effectiveness of interventions for at-risk patients. In summary, predictive analytics plays an increasingly important role in population health.

These tools are becoming an integral part of modern business and enable great success in marketing and customer service. The Interface component serves as the interaction point between the environment and users. Furthermore, the interface enables researchers to create new models, evaluation methods, guidelines, and benchmarks within the provided environment.

The authors would like to express their appreciation to each person who took part in this online survey. First, the measurement model was tested to examine the validity and reliability of the survey instruments. Table 2 shows that for all the instruments, the Cronbach’s α, Dijkstra-Henseler’s ρA, and composite reliability (CR) exceed 0.7, indicating acceptable internal reliability of the tools (Fornell and Larcker, 1981; Dijkstra and Henseler, 2015). Furthermore, the factor loadings of the instruments were all higher than the expected value of 0.7, and the average variance extracted (AVE) varied from 0.652 to 0.851, which is higher than the threshold value of 0.5 (Fornell and Larcker, 1981). Data from 398 participants were used to construct a partial least squares structural equation model (PLS-SEM). AI technologies can take over mundane, repetitive tasks, such as checking a claim’s status, and enabling the human staff to focus on more complex revenue cycle management objectives.

AI Chatbots Could Benefit Dementia Patients

For one thing, the USMLE is given in-person without test-takers having access to web-based tools to find answers, notes Alex J. Mechaber, MD, vice president of the USMLE at the National Board of Medical Examiners. Mechaber also points out that the chatbots did not take the full exam, but answered certain types of sample questions from various sources. Chatbots can produce summaries of just about anything studied in medical school, including biological functions, illnesses, and treatments. But the answers do not cite sources, leaving users responsible for verifying their accuracy or vulnerable to passing along errors.

However, successfully implementing predictive analytics requires high-quality data, advanced technology, and human oversight to ensure appropriate and effective interventions for patients. The second crucial requirement involves creating comprehensive human guidelines for evaluating healthcare chatbots with the aid of human evaluators. Healthcare professionals can assess the chatbot’s performance from the perspective of the final users, while intended users, such as patients, can provide feedback based on the relevance and helpfulness of answers to their specific questions and goals. As such, these guidelines should accommodate the different perspectives of the chatbot’s target user types. General-purpose human evaluation metrics have been introduced to assess the performance of LLMs across various domains5. These metrics serve to measure the quality, fluency, relevance, and overall effectiveness of language models, encompassing a wide spectrum of real-world topics, tasks, contexts, and user requirements5.

This hypothesis is supported by studies that achieved higher reliability and validity in the generation process by using methods that effectively ground LLM representations in external sources of knowledge [30, 32]. Consequently, the knowledgebase of Chat-GPT is contingent upon the timeliness and accuracy of its training data. For instance, GPT-3.5’s knowledge is constrained as it is founded on information available only up until September 2021.

Understanding the Role of Chatbots in Virtual Care Delivery

A recent study by Tata Consultancy Services (TCS) highlights the profound impact of AI on healthcare, illustrating how this technology is enhancing productivity, improving quality, and reshaping the future of medical care. As we continue to improve our understanding of AI and further our pursuit of innovation and discovery, it’s up to healthcare providers around the world to question how best to utilize the tools at their disposal. Already, the World Health Organization (WHO) has issued additional guidelines for safe and ethical AI use in the healthcare space — a continued effort that builds off their original 2021 guidelines but with added caution around large language models like ChatGPT and Bard. Factors affecting COVID-19 vaccine hesitancy included perceptions of vaccine importance, efficacy and safety, concerns about side effects, vaccine accessibility, and personal or religious beliefs15,16,17,18. Major concerns from seniors regarded the risk of serious adverse events following immunisation, such as deaths and complications due to old age and medical history19. For example, in Thailand, vaccines were provided by both the government and the private sector; however, results from cross-sectional surveys indicated that vaccination uptake among Thai people, especially among seniors, was low compared to other Southeast Asian countries20.

• Prior research has primarily emphasized the impact of rational considerations such as acceptability (Boucher et al., 2021), perceived utility (Nadarzynski et al., 2019), and performance expectancy (Huang et al., 2021), on individuals’ health chatbot adoption behavior.
• It is important to note that performance metrics may remain invariant concerning the three confounding variables (user type, domain type, and task type).
• AI has been used in healthcare settings to develop diagnostic tools and personalized treatment plans.
• One key issue of generative AI tools like chatbots is hallucination—cases where the AI confidently provides a false answer.
• The absence of formal collaboration between healthcare providers and chatbot developers inhibits widespread knowledge and utilization of these innovative tools among patients and medical professionals.
• The AI-powered chatbot offers health-related advice in eight languages, covering subjects such as healthy eating, mental health, cancer, heart disease and diabetes.

You can instruct a chatbot to urge a user to see a doctor when it obtains information about particular symptoms, chronic conditions or medical history details. In healthcare, hallucinations by AI chatbots can result in wrong diagnoses, mistreatment, and misinformation of patients. All of these can lead to lawsuits, fines and penalties—or even to the closure of a healthcare institution. Large Language Models are capable of generating highly convincing human-like responses and engaging in interactive conversations.

A survey of 2,000 conducted by the University of Arizona Health Sciences showed that 52 percent preferred consulting with real physicians over AI chatbots. But, importantly, the survey revealed that encouragement from their physicians could help patients overcome benefits of chatbots in healthcare their hesitation. Most physicians believe that chatbots are beneficial in scheduling medical appointments (78 percent), locating health clinics (76 percent), or providing medication information (71 percent), a survey that polled 100 physicians shows.

The impact of ChatGPT on mental health and mental healthcare service delivery is yet to be determined. That will be answerable with a far superior AI-guided program, research, robust regulatory and monitoring mechanisms, and integration of human intervention with ChatGPT. Public awareness of AI in health and medicine is still in the process of developing, yet even at this early stage, Americans make distinctions between the types of applications they are more and less open to. For instance, majorities say they would want AI-based skin cancer detection used in their own care and think this technology would improve the accuracy of diagnoses.

Research on whether people prefer AI over healthcare practitioners has shown mixed results depending on the context, type of AI system, and participants’ characteristics [107, 108]. Some surveys have indicated that people are generally willing to use or interact with AI for health-related purposes such as diagnosis, treatment, monitoring, or decision support [108,109,110]. However, other studies have suggested that people still prefer human healthcare practitioners over AI, especially for complex or sensitive issues such as mental health, chronic diseases, or end-of-life care [108, 111]. In a US-based study, 60% of participants expressed discomfort with providers relying on AI for their medical care. However, the same study found that 80% of Americans would be willing to use AI-powered tools to help manage their health [109].

You can foun additiona information about ai customer service and artificial intelligence and NLP. As AI technology continues to develop, we can expect to see even more innovative and effective ways to use AI to educate patients. Despite the increasing role of technology in healthcare, this study found that more people still prefer consultations with doctors over chatbots. However, when it comes to potentially embarrassing sexual symptoms, chatbots were more accepted and preferred by more participants than when considering other symptom categories.

AI has huge potential to empower patients by democratizing access to medical information. AI-driven tools — such as virtual assistants and health apps — can offer patients personalized educational resources, practical tips for managing their condition, and insights into how they can improve their overall wellbeing. Today, AI-powered chatbots can also provide patients with personalized reminders and support for sticking to their treatment plans. The World Economic Forum predicts AI may help automate diet recording,5 potentially increasing the accuracy of the records and easing the burden of tracking patients. While AI is not meant to, and cannot, replace the role of healthcare professionals, it can complement human skills by providing support and assistance with various medical tasks.

The researchers found that all of the chatbots had promising features, but none are yet likely to be effective at providing reliable, evidence-based information or emotional support. The authors recommended that the development and research into chatbot apps continue to increase, as it holds great potential to benefit dementia patients. Reputable cloud providers make substantial investments in security measures to aid healthcare firms in adhering to regulatory standards, such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States.

For example, a recent study confirmed that user emotions impact innovation evaluation and subsequent resistance behavior (Castro et al., 2020). Therefore, future research should consider the effects of factors such as individual emotions, cultural context, and social circumstances on individuals’ resistance behaviors. Drawing on the PWM, this study reveals the dual rational/irrational mediating mechanisms underlying people’s resistance to health chatbots. In particular, this study demonstrates that individuals’ perceived functional and psychological barriers may significantly influence their resistance intention, thereby increasing the likelihood of subsequent resistance behavioral tendency. Similarly, negative prototypes regarding health chatbots may increase resistance behavioral tendency through resistance intention and resistance willingness. Importantly, our results indicate that negative prototype perceptions regarding health chatbots have a greater impact on individuals’ resistance willingness and their subsequent resistance behavioral tendency than functional and psychological barriers.

Even still, they noted that ChatGPT could help fill a symptom checker void and act as an adjunct for providers. But that might be upended both by advances in artificial intelligence and consumer demand. The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

The three key configuration components consist of confounding variables, prompt techniques and parameters, and evaluation methods. Between-category relations occur when metrics from different categories exhibit correlations. Empathy often necessitates personalization, which can potentially compromise privacy and lead to biased responses. For both groups, the researchers worked to determine whether the patients would rather consult with an AI chatbot like ChatGPT or with a physician. President Joe Biden passed an executive order aimed to reduce some of the risks posed by AI.

However, the potential of AI chatbots is clear, and the pace of development means that despite challenges, such tools will assist many industries — especially those that involve conversations and sifting through conversations for vital information. Generative AI is significantly transforming customer service, with AI chatbots and virtual assistants now able to handle complex queries and offer personalized responses. However, it is crucial for organizations to monitor these chatbots closely to ensure they provide appropriate care.

Healthcare providers need to be aware of these shortcomings and their patients’ habits around online medical search and research. Using the PLS-SEM algorithm and bootstrapping resampling procedure, this study evaluated the path coefficients and significance of the proposed model. The explained variance (R2) and effect size (f2) were also estimated to test the model’s explanatory power and actual efficacy, respectively.

Based on application, the market is divided into symptoms check, medical & drug information assistance, appointment scheduling & monitoring, and other applications. Based on end user, the market is classified into healthcare providers, healthcare payers, patients, and other end users. The use of ChatGPT and ChatGPT-supported chatbot in educational training and clinical curricula should be explored, ensuring their integration aligns with the evolving needs of learners and practitioners. Research is needed to evaluate the chatbot program’s effectiveness, impact on learners and facilitators, and to identify potential issues or areas for improvement (Cascella et al., 2023; Kooli, 2023; Xue et al., 2023). Generating evidence-based protocols for the application of chatbot for mental healthcare is urgently required.

In research, AI has been used to analyze large datasets and identify patterns that would be difficult for humans to detect; this has led to breakthroughs in fields such as genomics and drug discovery. AI has been used in healthcare settings to develop diagnostic tools and personalized treatment plans. As AI continues to evolve, it is crucial to ensure that it is developed responsibly and for the benefit of all [5,6,7,8]. Artificial Intelligence (AI) is a rapidly evolving field of computer science that aims to create machines that can perform tasks that typically require human intelligence. AI includes various techniques such as machine learning (ML), deep learning (DL), and natural language processing (NLP). Large Language Models (LLMs) are a type of AI algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate, and predict new text-based content [1,2,3].