To all my budding surgeons: how cosy have you gotten with GPT lately? Is it honing your scalpel skills… or just letting you coast on autopilot while you daydream about how good the was.

This week, two fresh studies landed with very different takes on AI in medical procedures.

So, is AI in the operating theatre a force for good… or just a force for incompetence?

First up, a randomised controlled trial in from McGill University. They’ve built a fancy AI surgical teaching system called Intelligent Continuous Expertise Monitoring System (ICEMS).

Here’s how it works:

You practice surgery in VR.

The system measures everything from bleeding risk to healthy tissue injury risk to how aggressively you’re holding your bipolar forceps.

It spits out a surgical score from −1.00 (novice) to 1.00 (expert), comparing you against an established pro.

It instructs your practice giving you real time feedback.
87 students trained with either:
AI-only feedback,
AI read aloud by a human

Personalised AI-informed human coaching.

The key finding of this trial is that group 3 was the best. Achieving much higher scores across trials and the realistic task than group 1 and 2. Unfortunately, this trial did not have an intervention group that was human instruction alone. So we don't know if AI + Human > Human alone.

All in all though, we can see AI is a pretty good teacher.

On the other hand we have this study out of The Lancet Gastroenterology.

Since late 2021, four Polish endoscopy centres have been part of the ACCEPT trial, a real-world programme where AI systems assist in spotting polyps during colonoscopy.

The researchers wanted to know: if doctors get used to having AI spot polyps for them, does their own detection ability drop when the AI isn’t there?

They reviewed 1,442 colonoscopies before and after AI systems were rolled out across four endoscopy centres in the ACCEPT trial.

The research found:

Polyp detection in non-AI procedures dropped by 6% after regular AI exposure.

Adenoma detection rate(the gold-standard measure of colonoscopy quality) fell from 28.4% to 22.4%.

Experience didn’t protect: The decline occurred regardless of the endoscopist’s years of experience or specialty.

“To our knowledge this is the first study to suggest a negative impact of regular AI use on healthcare professionals’ ability to complete a patient-relevant task in medicine of any kind” - says one of the study authors.

So, what do we make of this?

These two studies are like the “angel and devil on your shoulder” when it comes to AI in healthcare. If there is one thing in common, human factors remain critical. In teaching, AI remains fine. In practice, the line is a bit more grey.

.