‘Behind the times’: Washington tries to catch up with AI’s use in healthcare
Lawmakers and regulators in Washington are starting to puzzle over how to regulate artificial intelligence in healthcare — and the AI industry thinks there’s a good chance they’ll mess it up.
“It’s an incredibly daunting problem,” said Dr. Robert Wachter, chair of the Department of Medicine at UC San Francisco. “There’s a risk we come in with guns blazing and overregulate.”
Already, AI’s impact on healthcare is widespread. The Food and Drug Administration has approved 692 AI products. Algorithms are helping to schedule patients, determine staffing levels in emergency rooms and even transcribe and summarize clinical visits to save physicians’ time. They’re starting to help radiologists read MRIs and X-rays. Wachter said he sometimes informally consults a version of GPT-4, a large language model from the company OpenAI, for complex cases.
The scope of AI’s impact — and the potential for future changes — means government is already playing catch-up.
“Policymakers are terribly behind the times,” Michael Yang, senior managing partner at OMERS Ventures, a venture capital firm, said in an email. Yang’s peers have made vast investments in the sector. Rock Health, a venture capital firm, says financiers have put nearly $28 billion into digital health firms specializing in artificial intelligence.
Many doctors and consumer advocates say that AI can’t live up to its hype.
One issue regulators are grappling with, Wachter said, is that, unlike drugs, which will have the same chemistry five years from now as they do today, AI changes over time. But governance is forming, with the White House and multiple health-focused agencies developing rules to ensure transparency and privacy. Congress is also flashing interest; the Senate Finance Committee held a hearing on AI in healthcare last week.
Along with regulation and legislation comes increased lobbying. CNBC counted a 185% surge in the number of organizations disclosing AI lobbying activities in 2023. The trade group TechNet has launched a $25-million initiative, including TV ad buys, to educate viewers on the benefits of artificial intelligence.
“It is very hard to know how to smartly regulate AI since we are so early in the invention phase of the technology,” Bob Kocher, a partner with venture capital firm Venrock who previously served in the Obama administration, said in an email.
Kocher has spoken to senators about AI regulation. He emphasizes some of the difficulties the healthcare system will face in adopting the products. Doctors — facing malpractice risks — might be leery of using technology they don’t understand to make clinical decisions.
An analysis of Census Bureau data from January by the consultancy Capital Economics found 6.1% of healthcare businesses were planning to use AI in the next six months, roughly in the middle of the 14 sectors surveyed.
Like any medical product, AI systems can pose risks to patients, sometimes in a novel way. One example: They might make things up.
Newsom’s office released a report about generative AI on Tuesday, outlining potential ways state government could harness the benefits of technology.
Wachter recalled a colleague who, as a test, assigned OpenAI’s GPT-3 to write a prior authorization letter to an insurer for a purposefully “wacky” prescription: a blood thinner to treat a patient’s insomnia.
But the AI “wrote a beautiful note,” he said. The system so convincingly cited “recent literature” that Wachter’s colleague briefly wondered whether she’d missed a new line of research. It turned out the chatbot had fabricated its claim.
There’s a risk of AI magnifying bias already present in the healthcare system. Historically, people of color have received less care than white patients. Studies show, for example, that Black patients with fractures are less likely to get pain medication than white ones. This bias could get set in stone if artificial intelligence is trained on that data and subsequently acts on it.
Research into AI deployed by large insurers has confirmed that has happened. But the problem is more widespread. Wachter said UCSF tested a product to predict no-shows for clinical appointments. Patients who are deemed unlikely to show up for a visit are more likely to be double-booked.
The test showed that people of color were more likely not to show. Whether or not the finding was accurate, “the ethical response is to ask, why is that, and is there something you can do,” Wachter said.
One of the amazing (and scary) things about artificial intelligence programs is that in learning to mimic their human masters so perfectly, these wonders of computer software hold up a mirror to patterns of behavior we engage in every day but may not even notice.
Hype aside, those risks will likely continue to grab attention over time. AI experts and FDA officials have emphasized the need for transparent algorithms, monitored over the long term by human beings — regulators and outside researchers. AI products adapt and change as new data is incorporated. And scientists will develop new products.
Policymakers will need to invest in new systems to track AI over time, said University of Chicago Provost Katherine Baicker, who testified at the Senate Finance Committee hearing. “The biggest advance is something we haven’t thought of yet,” she said in an interview.
KFF Health News, formerly known as Kaiser Health News, is a national newsroom that produces in-depth journalism about health issues.
