Your Doctor’s AI Is Listening. Has Anybody Checked Whether It Sees You?

More than 80% of doctors now use AI with patients. For Black and brown patients, the tools have a track record worth knowing about.

Next time you’re in the exam room, your doctor may ask if it’s OK to turn on the AI. It’ll take notes, the doctor says. Help them pay attention. Most people say yes. Nobody wants to be the patient who holds up the visit.

Here’s what you’re saying yes to, and why it matters more if you’re Black or brown.

It’s Already Everywhere

More than 80% of physicians now use some kind of artificial intelligence in their work with patients, Dr. Barbara Rubino, associate chief medical officer at Covered California, wrote in a San Francisco Chronicle op-ed this week. Eighty-four percent of health insurers use it too, she wrote, for things like paying claims and deciding prior authorizations. A Blue Cross Blue Shield report she cited found hospitals and insurers using AI against each other, one side to bill more, the other to flag and deny, adding nearly $1 billion in health care spending from 2023 to 2025.

Her conclusion: there’s little evidence it’s helping patients. Two-thirds of adults have low trust in how the health system uses AI, she wrote.

Black patients have been here before.

The Track Record

AI learns from old data. In American medicine, old data carries old ideas about race.

In 2019, researchers found that a widely used hospital algorithm, applied to tens of millions of patients, steered extra care toward white patients over Black patients who were just as sick. The program used past health spending to predict who needed help. Black patients had less spent on them, so the algorithm decided they needed less.

In 2023, Stanford researchers tested popular AI chatbots, including OpenAI’s ChatGPT, Google’s Bard and Anthropic’s Claude, on medical questions about race. The bots repeated debunked, race-based ideas about kidney function, lung capacity and skin thickness, NBC News reported. These are the same myths doctors have spent years trying to scrub out of medicine.

Researchers have also found that racial bias built into common lung-function testing software likely meant fewer Black patients got care for breathing problems, according to NBC.

None of that means the AI in your doctor’s office is biased. It means nobody should assume it isn’t.

Why It Hits Harder Here

Black and brown patients already walk into the system with less trust and, too often, worse outcomes. Put a tool between patient and doctor that was trained mostly on somebody else’s charts, and the gap doesn’t close by itself. If an insurer’s software flags your claim, you may never see what it was looking at.

And the power runs one way. Rubino describes a colleague who felt pressured to say yes to the AI note-taker just to keep her visit moving. Now picture that question in an emergency room, or asked of an elder who doesn’t want to seem difficult.

What California Already Requires

California has two laws on the books since January 2025:

• AB 3030: If a clinic or doctor’s office uses generative AI to write you a message about your care, it has to tell you, and it has to tell you how to reach a real person, unless a provider reviewed the message first.
• SB 1120: A health plan can’t let software alone deny your care as medically unnecessary. A licensed physician or qualified health professional has to make the call. The tools have to be applied fairly, can’t discriminate, and are open to state audits.

What You Can Do

• You can say no. Ask what the AI records, where it goes and who sees it. If you’re not comfortable, decline the note-taker.
• Ask a human. If you get a message that reads like a machine wrote it, ask to speak with your provider.
• Appeal denials. If your insurer denies care, ask whether a physician reviewed it. Under California law, one has to.
• Use the tools yourself. Rubino suggests patients use AI to prepare questions, review bills and draft appeals. Just double-check anything it tells you with your doctor.

The machines are already in the room. The least they can do is see everybody in it.

This story draws on a San Francisco Chronicle op-ed by Dr. Barbara Rubino, NBC News reporting on a Stanford study, and published summaries of California law.

Photo: A doctor explains an X-ray to patients. Rhoda Baer/National Cancer Institute via Wikimedia Commons, public domain.