According to a research article published in JAMA, ChatGPT responded to inquiries about cardiovascular disease prevention "mostly appropriately." 25 questions were created by researchers to cover key ideas in heart disease prevention, such as risk factor counseling, test findings, and prescription information. They asked the AI chatbot the questions three times, and a clinician rated each set of answers.
If the chatbot's responses varied in each set, the doctor assessed the responses as appropriate, improper, or unreliable. The responses were also assessed in two different contexts: first, as patient-facing platforms, and second, as draft replies to patient messages given to a doctor for review, reports MobiHealth News.
In the study, it was discovered that 21 of the 25 questions were rated as appropriate in both scenarios, while only four were improper. Three of the four sets of wrong responses contained all three incorrect answers, while one set contained just one such response.
The analysis's limitations were recognized by the researchers. Cardiovascular disease prevention cannot be covered in just 25 questions at this time because the chatbot is not yet intended for medical usage. They advised using more reviewers to assess responses in subsequent studies or creating a formal grading system that didn't rely as heavily on a clinician's opinion.
"Findings suggest the potential of interactive AI to assist clinical workflows by augmenting patient education and patient-clinician communication around common CVD prevention queries. For example, such an application may provide conversational responses to simple queries on informational platforms or create automated draft responses to patient electronic messages for clinicians," they wrote. "Whether these approaches can improve readability should be explored, because prior work has indicated low readability of certain online patient educational materials for CVD prevention."
Many businesses promote chatbots for use in the healthcare industry. The company Wysa raised $20 million last summer. Wysa creates a chatbot that leads customers through cognitive behavioral therapy for issues including depression, anxiety, and stress.
Woebot Health, a business that also develops chatbots for mental health issues, received a $9.5 million investment from Leaps by Bayer about a year ago. In 2021, Woebot's postpartum depression digital therapy was given FDA Breakthrough Device Designation.