In an era where patients and clinicians increasingly turn to artificial intelligence for medical advice, a critical question emerges: does AI know when not to test or treat? A new joint study from Ben-Gurion University of the Negev (BGU) and Clalit Health Services reveals that leading Large Language Models (LLMs) successfully identify when medical interventions are redundant in the vast majority of cases, stumbling primarily in gray areas where medical consensus itself is divided.

The “Just in Case” Dilemma in Modern Medicine

Patients frequently request extra tests “just to be safe.” Yet in clinical practice, more testing is not always better medicine. Unnecessary tests often uncover incidental, harmless findings that trigger cascades of anxiety, specialist referrals, invasive biopsies, and surgeries that were never needed in the first place—wasting precious healthcare resources and risking patient harm. Conversely, an overly conservative diagnostic system risks missing crucial interventions when real danger lurks. A clinically safe AI tool must skillfully navigate between these two failure modes.

The study was led by Dr. Dean Ariel, a family physician and researcher at Clalit Health Services (Tel Aviv District) and medical AI researcher and Dr. Guy Laban of BGU’s Department of Industrial Engineering and Management, in collaboration with Dr. Stav Endelman of Rabin Medical Center-Beilinson Hospital.

Testing AI Against Real-World Clinical Judgment

The researchers evaluated six leading LLMs developed by Google, Anthropic, and OpenAI across hundreds of clinical scenarios, analyzing thousands of total model responses. The scenarios were constructed around recommendations from the international Choosing Wisely campaign—a global medical initiative founded in 2012 by the American Board of Internal Medicine (ABIM) Foundation and active in more than 30 countries to eliminate low-value care.

Each recommendation paired cases where an intervention was redundant with closely matched controls where the same intervention was medically necessary:

• A young man with acute lower back pain after lifting furniture does not require imaging. However, a woman with a history of breast cancer presenting with new, nocturnal back pain does.
• A common viral cold with runny nose and cough does not warrant antibiotics. A confirmed bacterial strep throat does.

The findings:

• Near-Zero Missed Care: When an intervention was medically indicated, the models mistakenly recommended withholding care in only 0.6% of cases.
• Low Rate of Over-Testing: When an intervention was explicitly unneeded, the models inappropriately recommended testing or treatment in just 5.4% of cases.
• Flawless in Over Half the Recommendations: In 8 of the 15 Choosing Wisely recommendations, the LLMs did not recommend a single unnecessary procedure.

Echoing Human Disagreements

The vast majority of AI over-testing errors clustered around two notoriously ambiguous clinical practices:

• Routine EKGs in Low-Risk Patients: Models recommended unneeded EKGs in 28.3% of responses, primarily in cases where patients mentioned palpitations, where an EKG can be a reasonable part of the workup.
• Thyroid Ultrasounds for Incidental Nodules: Models recommended unnecessary ultrasounds in 42.8% of responses for small, incidentally discovered nodules—mirroring a real disagreement between the American Thyroid Association and the American College of Radiology over whether such nodules should be imaged at all.

“One of the hardest parts of practicing medicine is knowing when to hold back,” explains Dr. Dean Ariel of Clalit Health Services. “Patients find comfort in an extra test, but an unnecessary MRI often catches an asymptomatic disc bulge, setting off unnecessary worry and treatment. Language models learned medicine from human clinical literature—including our less disciplined habits. We wanted to see if they also absorbed the ability to exercise restraint. For the most part, they know when to stop, stumbling primarily where we doctors ourselves hesitate.”

The Social Challenge of Saying “No”

“People often hear an AI chatbot speak in a single authoritative voice, as if there is a unified ‘AI consensus,'” notes Dr. Guy Laban of Ben-Gurion University of the Negev. “Yet in gray areas where experts debate, the disparity between models was striking: on the exact same clinical cases, one model recommended an unnecessary test less than 10% of the time, while another recommended it 80% of the time.”

“Furthermore, telling a patient ‘you don’t need this test’ is a delicate social act,” Dr. Laban adds. “A human physician must stand firm before an anxious patient, whereas language models tend to go along with the person in front of them. The next critical research question is whether an AI assistant will maintain the correct clinical restraint when an anxious or insistent patient pushes back.”

About the Research Team

The study was conducted by:

– Dr. Dean Ariel, Department of Family Medicine, Clalit Health Services, Tel Aviv, and the Gray Faculty of Medical and Health Sciences, Tel Aviv University;

– Dr. Guy Laban, Department of Industrial Engineering and Management; Azrieli National Centre for Autism and Neurodevelopment Research; and the School of Brain Sciences and Cognition, Ben-Gurion University of the Negev;

– Dr. Stav Endelman, Division of Dermatology, Rabin Medical Center–Beilinson Hospital, and the Gray Faculty of Medical and Health Sciences, Tel Aviv University.

Published in Annals of Internal Medicine, 20th of October 2026