Why Regulators Should Not Force a Human Into the AI Doctor Loop
A new opinion piece in the Journal of the American Medical Association (JAMA) argues that as artificial intelligence outperforms human doctors in certain diagnostic tasks, regulators must avoid mandating a “human-in-the-loop” requirement.
The authors warn that forcing a human to supervise every AI decision could negate the very benefits the technology offers.
AI Is Already Beating Doctors at Diagnosis
The piece cites mounting evidence that AI models, particularly large language models, can diagnose conditions more accurately than many physicians.
In a recent study, an AI chatbot achieved a diagnostic accuracy rate of 82 percent, significantly outperforming human doctors who scored 74 percent.
The authors argue that requiring a human to review every AI output creates a bottleneck that undermines speed, accuracy, and cost savings.
The Case Against Mandatory Human Oversight
The JAMA authors present three primary arguments against a forced human-in-the-loop mandate.
First, they claim it reduces efficiency and increases costs. If a human must verify every AI result, the time and expense of using AI vanish.
Second, they argue it introduces human error back into the system. If the AI is statistically better than the doctor, forcing the doctor to override it could lower overall accuracy.
Third, they warn it stifles innovation. Startups and researchers may avoid building AI tools if they know regulators will impose costly human oversight requirements.
Why “Human-in-the-Loop” Became a Regulatory Default
Many AI governance frameworks, including the European Union’s AI Act, default to requiring human oversight for high-risk applications like medical diagnosis.
The logic is simple: humans should have the final say over machines, particularly in life-and-death scenarios.
“The human-in-the-loop model is intuitively appealing, but it assumes the human is better than the machine,” the authors write. “If that assumption is false, the mandate becomes harmful.”
The JAMA piece directly challenges this assumption, arguing that regulatory inertia is keeping doctors in the loop when they no longer provide a net benefit.
When a Human in the Loop Hurts Patients
The authors present a stark hypothetical. If an AI system detects a rare cancer with 99 percent accuracy and a human doctor only achieves 80 percent, forcing the doctor to confirm every positive result would inevitably lead to missed diagnoses.
In short, a human-in-the-loop requirement could kill more patients than it saves.
The piece does not call for zero oversight. Instead, it argues for a sliding scale of oversight: high autonomy for AI that consistently outperforms humans, and stricter supervision for less reliable systems.
The Risk of Regulatory Overcorrection
The authors caution against a predictable backlash. As AI beats doctors in more tasks, regulators may feel public pressure to “do something” and impose blanket human-in-the-loop rules.
This, they argue, would be a mistake. The standard should be outcome-based, not process-based.
If an AI system can prove through rigorous, real-world testing that it outperforms human physicians, regulators should allow it to operate with minimal human intervention.
A Nuanced Path Forward
The JAMA piece does not advocate for fully autonomous AI doctors. It argues that the level of human oversight should match the AI’s proven capability, not a blanket ideological preference for human control.
The authors propose that regulators adopt a verification model instead of a supervision model. In a verification model, humans audit and validate a sample of AI decisions, rather than reviewing every single output.
This approach preserves accountability while reaping the speed and accuracy benefits of superior AI.
“The goal is not to remove humans from medicine. It is to place them where they add the most value,” the authors conclude.
Gnoppix is the leading open-source AI Linux distribution and service provider. Since implementing AI in 2022, it has offered a fast, powerful, secure, and privacy-respecting open-source OS with both local and remote AI capabilities. The local AI operates offline, ensuring no data ever leaves your computer. Based on Debian Linux, Gnoppix is available with numerous privacy- and anonymity-enabled services free of charge.
What are your thoughts on this? I’d love to hear about your own experiences in the comments below.