{"id":1720,"date":"2023-09-12T10:00:40","date_gmt":"2023-09-12T10:00:40","guid":{"rendered":"https:\/\/healthynewsusa.com\/?p=1720"},"modified":"2023-09-12T10:00:44","modified_gmt":"2023-09-12T10:00:44","slug":"dr-google-meets-its-match-dr-chatgpt","status":"publish","type":"post","link":"https:\/\/healthynewsusa.com\/?p=1720","title":{"rendered":"\u2018Dr. Google\u2019 Meets Its Match: Dr. ChatGPT"},"content":{"rendered":"
\n

As a fourth-year ophthalmology resident at Emory University School of Medicine, Riley Lyons\u2019 biggest responsibilities include triage: When a patient comes in with an eye-related complaint, Lyons must make an immediate assessment of its urgency.<\/p>\n

He often finds patients have already turned to \u201cDr. Google.\u201d Online, Lyons said, they are likely to find that \u201cany number of terrible things could be going on based on the symptoms that they\u2019re experiencing.\u201d<\/p>\n

So, when two of Lyons\u2019 fellow ophthalmologists at Emory came to him and suggested evaluating the accuracy of the AI chatbot ChatGPT in diagnosing eye-related complaints, he jumped at the chance.<\/p>\n

In June, Lyons and his colleagues reported in medRxiv, an online publisher of health science preprints, that ChatGPT compared quite well<\/a> to human doctors who reviewed the same symptoms \u2014 and performed vastly better than the symptom checker on the popular health website WebMD. And despite the much-publicized \u201challucination\u201d problem known to afflict ChatGPT \u2014 its habit of occasionally making outright false statements \u2014 the Emory study reported that the most recent version of ChatGPT made zero \u201cgrossly inaccurate\u201d statements when presented with a standard set of eye complaints.<\/p>\n

\n

\n\t\t
\n\t\t\tEmail Sign-Up\t\t<\/a>
\n\t<\/h4>\n

\n\t\tSubscribe to KFF Health News’ free Morning Briefing.\t<\/p>\n<\/div>\n

The relative proficiency of ChatGPT, which debuted in November 2022, was a surprise to Lyons and his co-authors. The artificial intelligence engine \u201cis definitely an improvement over just putting something into a Google search bar and seeing what you find,\u201d said co-author Nieraj Jain, an assistant professor at the Emory Eye Center who specializes in vitreoretinal surgery and disease.<\/p>\n

But the findings underscore a challenge facing the health care industry as it assesses the promise and pitfalls of generative AI, the type of artificial intelligence used by ChatGPT: The accuracy of chatbot-delivered medical information may represent an improvement over Dr. Google, but there are still many questions about how to integrate this new technology into health care systems with the same safeguards historically applied to the introduction of new drugs or medical devices.<\/p>\n

The smooth syntax, authoritative tone, and dexterity of generative AI have drawn extraordinary attention from all sectors of society, with some comparing its future impact to that of the internet itself. In health care, companies are working feverishly to implement generative AI in areas such as radiology and medical records.<\/p>\n

When it comes to consumer chatbots, though, there is still caution, even though the technology is already widely available \u2014 and better than many alternatives. Many doctors believe AI-based medical tools should undergo an approval process similar to the FDA\u2019s regime for drugs, but that would be years away. It\u2019s unclear how such a regime might apply to general-purpose AIs like ChatGPT.<\/p>\n

\u201cThere\u2019s no question we have issues with access to care, and whether or not it is a good idea to deploy ChatGPT to cover the holes or fill the gaps in access, it\u2019s going to happen and it\u2019s happening already,\u201d said Jain. \u201cPeople have already discovered its utility. So, we need to understand the potential advantages and the pitfalls.\u201d<\/p>\n

The Emory study is not alone in ratifying the relative accuracy of the new generation of AI chatbots. A report published in Nature<\/a> in early July by a group led by Google computer scientists said answers generated by Med-PaLM, an AI chatbot the company built specifically for medical use, \u201ccompare favorably with answers given by clinicians.\u201d<\/p>\n

AI may also have better bedside manner. Another study, published in April by researchers from the University of California-San Diego and other institutions, even noted that health care professionals rated ChatGPT answers as more empathetic<\/a> than responses from human doctors.<\/p>\n

Indeed, a number of companies are exploring how chatbots could be used for mental health therapy, and some investors in the companies are betting that healthy people might also enjoy chatting and even bonding with an AI \u201cfriend.\u201d The company behind Replika, one of the most advanced of that genre, markets its chatbot as, \u201cThe AI companion who cares. Always here to listen and talk. Always on your side.\u201d<\/p>\n

\u201cWe need physicians to start realizing that these new tools are here to stay and they\u2019re offering new capabilities both to physicians and patients,\u201d said James Benoit, an AI consultant. While a postdoctoral fellow in nursing at the University of Alberta in Canada, he published a study in February reporting that ChatGPT significantly outperformed<\/a> online symptom checkers in evaluating a set of medical scenarios. \u201cThey are accurate enough at this point to start meriting some consideration,\u201d he said.<\/p>\n

Still, even the researchers who have demonstrated ChatGPT\u2019s relative reliability are cautious about recommending that patients put their full trust in the current state of AI. For many medical professionals, AI chatbots are an invitation to trouble: They cite a host of issues relating to privacy, safety, bias, liability, transparency, and the current absence of regulatory oversight.<\/p>\n

The proposition that AI should be embraced because it represents a marginal improvement over Dr. Google is unconvincing, these critics say.<\/p>\n

\u201cThat\u2019s a little bit of a disappointing bar to set, isn\u2019t it?\u201d said Mason Marks, a professor and MD who specializes in health law at Florida State University. He recently wrote an opinion piece<\/a> on AI chatbots and privacy in the Journal of the American Medical Association. \u201cI don\u2019t know how helpful it is to say, \u2018Well, let\u2019s just throw this conversational AI on as a band-aid to make up for these deeper systemic issues,\u2019\u201d he said to KFF Health News.<\/p>\n

The biggest danger, in his view, is the likelihood that market incentives will result in AI interfaces designed to steer patients to particular drugs or medical services. \u201cCompanies might want to push a particular product over another,\u201d said Marks. \u201cThe potential for exploitation of people and the commercialization of data is unprecedented.\u201d<\/p>\n

OpenAI, the company that developed ChatGPT, also urged caution.<\/p>\n

\u201cOpenAI\u2019s models are not fine-tuned to provide medical information,\u201d a company spokesperson said. \u201cYou should never use our models to provide diagnostic or treatment services for serious medical conditions.\u201d<\/p>\n

John Ayers, a computational epidemiologist who was the lead author of the UCSD study, said that as with other medical interventions, the focus should be on patient outcomes.<\/p>\n

\u201cIf regulators came out and said that if you want to provide patient services using a chatbot, you have to demonstrate that chatbots improve patient outcomes, then randomized controlled trials would be registered tomorrow for a host of outcomes,\u201d Ayers said.<\/p>\n

He would like to see a more urgent stance from regulators.<\/p>\n

\u201cOne hundred million people have ChatGPT on their phone,\u201d said Ayers, \u201cand are asking questions right now. People are going to use chatbots with or without us.\u201d<\/p>\n

At present, though, there are few signs that rigorous testing of AIs for safety and effectiveness is imminent. In May<\/a>, Robert Califf, the commissioner of the FDA, described \u201cthe regulation of large language models as critical to our future,\u201d but aside from recommending that regulators be \u201cnimble\u201d in their approach, he offered few details.<\/p>\n

In the meantime, the race is on. In July, The Wall Street Journal reported that the Mayo Clinic was partnering with Google<\/a> to integrate the Med-PaLM 2 chatbot into its system. In June, WebMD announced<\/a> it was partnering with a Pasadena, California-based startup, HIA Technologies Inc., to provide interactive \u201cdigital health assistants.\u201d And the ongoing integration of AI into both Microsoft\u2019s Bing and Google Search suggests that Dr. Google is already well on its way to being replaced by Dr. Chatbot.<\/p>\n

This article was produced by <\/em>KFF Health News<\/em><\/a>, which publishes <\/em>California Healthline<\/em><\/a>, an editorially independent service of the <\/em>California Health Care Foundation<\/em><\/a>.<\/em>\u00a0<\/p>\n