Algorithm Has No Medical License: Who Answers When AI Gets It Wrong?
By André Leite and Vinícius Lain, authors of AI in Healthcare.
In Brazil, every physician carries a registration number with their professional council, the CRM, the Brazilian medical license number, which exists precisely so that there is individual, traceable, enforceable responsibility for every clinical decision. An artificial intelligence algorithm has no CRM. And that gap, who answers when the AI gets it wrong, has become one of the most urgent and least resolved legal debates in contemporary medicine.
Consider a simple scenario: a diagnostic support system recommends, with high confidence, a course of action that turns out to be wrong, and the patient is harmed. Who is responsible? The physician who followed the suggestion without questioning it? Is the physician still responsible if the institution, in practice, encouraged or required the use of that tool? The hospital that adopted the system without validating it for its patient population? The vendor that developed and sold a flawed product? Each jurisdiction and each concrete case is still working out the answer, and today there is no definitive legal consensus, in Brazil or in most of the world.
There is an uncomfortable but useful historical analogy for thinking about this kind of distributed responsibility: the Nuremberg trials, which confronted head-on the question of how far "I was only following orders" or "following the system" excuses an individual from responsibility for their own decisions. This is obviously not a comparison of severity. It is a recognition that history has already taught us, painfully, that handing judgment entirely to an outside authority, human or algorithmic, does not relieve the person who carries out the final action of responsibility for it.
In clinical practice, this translates into a rule that should be obvious but needs to be said plainly: no artificial intelligence tool replaces the physician's final clinical judgment, or the responsibility that comes with it. AI can suggest, alert, and prioritize, but the signature, in both the literal and figurative sense, remains human.
Serious healthcare institutions are already building clear protocols for use: when to follow the algorithm's suggestion, when to question it, and how to document that decision. They are doing it precisely so that this responsibility doesn't sit in a legal limbo on the day when, inevitably, something goes wrong.
Keep reading
Ethics and Regulation
Who Is Responsible When Artificial Intelligence Makes a Mistake in Medicine?
When an AI takes part in a medical decision and something goes wrong, responsibility does not disappear. Understand the role of the physician, the institution, the vendor and governance.
Ethics and Regulation
The Game of Thrones of Health Data: Algorithmic Bias Is Inequality Disguised as Science
How algorithms trained on unequal data reproduce and amplify inequality, and why governance and explainability (XAI) are not optional.
AI in Medicine
Will Artificial Intelligence Replace Doctors?
Artificial intelligence is likelier to transform medical work than replace it. Learn what the machine can take over, and what remains essentially human.