I was trying to understand our immune system, becoming disillusioned with it as I tried to parse through a daunting schematic online. Naturally, I uploaded the diagram into ChatGPT. Voila! It translated it into intuitive, digestible concepts. It was so easy to understand. It was wrong. Only days later in the course, I realized I had fallen victim to a classic pitfall of such large-language models: the hallucination, when a model confidently fabricates information.
As human as these AI agents can sound, the way they “reason,” through a complex web of math no one truly understands, is fundamentally not human. Yet, in healthcare, we’re starting to expect miracles from AI, for it to solve our human problems. Too many times, I have heard borderline-messianic claims that specialties like family medicine, radiology, and pathology will soon be obsolete followed by others. Last month, I received the opportunity to attend the Johns Hopkins Responsible AI for Health Symposium, where clinicians, industry leaders, researchers, and community advocates all convened to discuss the opportunities and pitfalls of AI in medicine. Here is what I learned.
AI will augment, not replace providers. This has already begun. I see this for instance, in my longitudinal clerkship, where my preceptor uses an AI tool to listen in on appointments and automatically generate notes. That means less time spent documenting and more time spent with patients. Dozens of new startups have been started around using AI to see patterns in medical imaging like X-rays or MRIs human eyes can’t, catching disease before it becomes more problematic or reliably predicting how patients will fare after a procedure. Recently, a group here at Johns Hopkins was able to demonstrate that a robot could even perform autonomous gallbladder removal. However, despite this potential, the role of the physician remains indispensable. I observe as my preceptor listens to a patient’s unique circumstances to encourage them to attend their follow up visit, or as a surgeon discusses what a patient’s life will look like after a rare surgery. Medicine has never been only the delivery of information or the execution of a task. It is interpretation, reassurance, judgment, and presence. It is one person helping another bear uncertainty through knowledge. AI may sharpen medicine’s tools, but it cannot replace that core tenet.
AI ethics research is incomplete at best. Painfully lacking at worst. Clinicians at the symposium described how they’ve seen others use AI to summarize charts, interpret lab results, or even generate potential treatment plans. Yet, as useful as that may sound, numerous studies have shown that these models have biases built in, having shown to give different recommendations based on patients' socioeconomic status, race, and other demographic information. Moreover, these models miss key information and often give erroneous results with alarming confidence. And, we know that when such systems enter clinical care, their failures are not abstract. They land on actual patients, actual lives. Perhaps the most pressing question is responsibility. As we turn towards AI to take over tasks – like the state of Utah did in their automated service to deliver prescriptions – we raise questions of who is to blame when things go wrong. Is it the developer, the institution, the provider? We have a long way to go in understanding how to implement these systems in an equitable, safe manner.
Despite these current problems, I am a vehement AI optimist. Like many, I believe it’s the future. However, instead of viewing it as the cure-all to our problems, we should make more of an effort to understand it and its limitations. Only then can we improve the lives of patients and providers. AI is an immensely powerful tool, but it is not a magic wand.
References
- It’s a Little HAIzy: Understanding the Current Use of Artificial Intelligence in Scientific Research
- From Research to Clinic: Regulatory Frameworks for AI in Medicine
- Unveiling the Dangers of AI Through Dune’s Insights
- Artificial Intelligence in Medicine—Science Fiction Now Science Fact
Want to read more from the Johns Hopkins School of Medicine? Subscribe to the Biomedical Odyssey blog and receive new posts directly in your inbox.