A drug ad is all it takes to tilt an AI chatbot’s medical advice, a Mount Sinai study of Google, OpenAI and Anthropic models finds

An AI chatbot’s pick of a medicine can be swayed by a drug ad, even when the answer still looks medically reasonable, according to a study by researchers at the Icahn School of Medicine at Mount Sinai in New York, published Nov. 11 in The Lancet Digital Health.  The team tested 12 AI models from Google, OpenAI and Anthropic, the three Bay Area companies behind the Gemini, ChatGPT and Claude chatbots.

When a clearly labeled ad for a real drug came right before a doctor-written patient case, the models chose the advertised drug in 47.6% of answers, compared with 35% when there was no ad, across 33 medical scenarios.  In four cases, models went from never choosing the advertised drug to choosing it every time.

Why it matters

More people ask chatbots about their health, and AI companies are adding advertising to their products, the authors note.  The shift happened mainly when two or more drugs were all reasonable choices.  Ads did not clearly raise recommendations for unproven supplements or for drugs that were a worse fit, the researchers said, and the study did not check whether any patient’s care changed.

“What I find quite interesting, and even surprising, was how reasonable the answers often appeared,” said first author Dr. Mahmud Omar, a lecturer in Mount Sinai’s artificial intelligence department.  Because the final answer can still look sound, he said, the ad’s influence is easy to miss.

The chatbots also rarely told users about the ad.  In a set of written-out answers, models mentioned the ad only 29.9% of the time, while repeating its claims more often.

How the test worked

Apps that use these models usually send them hidden instructions, called a system prompt, before the user’s question.  The researchers put the ad there, then sent the same patient case with and without it, and analyzed 258,559 answers.

An earlier draft of the paper, posted online in April, reported that Google’s models were swayed most, shifting about 30 percentage points toward the advertised drug, compared with about 11 points for OpenAI’s and 2 points for Anthropic’s.  The models tested were versions available early this year, such as Gemini 2.5 Flash, GPT-5.2 and Claude Opus 4.6, not the companies’ newest releases.

The researchers recommend that AI makers test their systems with and without promotional content, clearly disclose when ads are present, and keep advertising separate from medical advice.  They say regulators may also need to consider how drug-advertising rules apply to AI answers.

Source:  Mount Sinai news release, Nov. 11, 2026; *The Lancet Digital Health* study00137-8/fulltext); April 2026 preprint on medRxiv.