The Elevator Operator Syndrome: Why Trust Matters More Than the Algorithm
By André Leite and Vinícius Lain, authors of AI in Healthcare.
When automatic elevators were introduced, able to run perfectly well without a human operator, many buildings kept an elevator operator on staff for years. Technology had already made the job technically obsolete, but the human presence was still needed for people to feel safe enough to actually use the elevator. The technology worked. Trust just hadn't caught up.
The same dynamic plays out, with far more serious consequences, in the adoption of artificial intelligence in healthcare. An algorithm can have flawless clinical validation and outperform the average physician's judgment on a specific task, and still be rejected, ignored, or used only superficially by the care team. The cause isn't a technical failure. The trust needed for real behavior change was never built. That is why the "champion" physician, a professional respected by peers who tests, validates, and advocates for the new tool from inside the profession, is often more decisive for the success of an implementation than any technical feature of the system.
A Cold War episode shows the flip side of the same coin. In 1983, Soviet officer Stanislav Petrov received an alert from the missile detection system indicating an American nuclear attack under way. Protocol said to report it immediately, which would likely have triggered nuclear retaliation. Petrov used human judgment against the automated system's recommendation, concluded it was a false alarm, and he was right. His decision to question the machine instead of following it blindly probably prevented a nuclear war.
The lesson for healthcare is not "always distrust technology" or "always trust it." It's that appropriate trust has to be calibrated, trained, and revisited constantly, never automatic in either direction. That takes real investment in training, in time for the team to understand how and why the system reaches its recommendations, and in safe institutional space for a professional to say "I think the algorithm is wrong here" without fear of being seen as resistant to innovation.
Technology mature enough to work on its own will still need, for a good while, people mature enough to know when to trust it, and when not to.
Keep reading
Doctors and AI
How Can AI Give Doctors Their Time Back?
Ambient listening, record summarization and document automation can reduce the invisible work that keeps doctors away from patients. Learn how to do it safely.
Doctors and AI
The Two Patients: Who Are You Really Treating?
Abraham Verghese's "iPatient" concept: the patient of data on the screen can steal attention from the real patient. See what this has to do with using AI in the consultation.
Doctors and AI
Who Stole Eye Contact From the Medical Consultation, and How AI Can Give It Back
Ambient listening: the technology that promises to give doctors back the eye contact the electronic record took out of the consultation.