The UK’s National Commission into the Regulation of AI in Healthcare has proposed 44 recommendations to create a new framework for overseeing AI tools in medical settings. The report, released by an independent body set up by the Medicines and Healthcare products Regulatory Agency (MHRA), argues that current rules—designed for traditional medical devices—fail to address the unique risks of AI systems.
Why existing rules fall short
AI-enabled medical tools differ fundamentally from conventional devices. Unlike static products, AI systems evolve rapidly through updates, and their performance depends on factors like data quality, user workflows, and the specific healthcare environment where they’re deployed. The commission’s report highlights that current oversight mechanisms don’t account for these variables.
The recommendations focus on three core areas: lifecycle regulation, system-wide responsibility, and trust, transparency, and predictability. The goal is to establish oversight that adapts to AI’s dynamic nature while ensuring patient safety. For example, the report suggests that regulatory approval could be granted in stages, allowing AI tools to enter the market under controlled conditions while developers gather real-world evidence on safety and effectiveness.
One key challenge is defining when an AI product should be classified as a medical device and how much regulatory scrutiny it requires. The commission proposes that oversight levels should align with the risk to patients and the intended purpose of the tool. Unlike traditional devices, AI’s purpose isn’t just tied to manufacturer claims; it also depends on how the system is designed to function in practice. The report calls for clearer guidance to prevent inconsistencies between a device’s design and its promotional materials.
Real-world monitoring and international alignment
The commission emphasizes the need for real-world data to track AI performance after deployment. Regulators should use testing environments where developers and oversight bodies can evaluate new technologies before widespread adoption. This approach would help identify issues early, such as disparities in how AI performs across different patient groups.
The report also stresses the importance of international regulatory harmonization. The MHRA is urged to create recognition pathways with other global regulators, allowing AI medical devices to meet multiple standards efficiently. These pathways should include review points and flexibility to adapt to emerging risks while encouraging innovation and market growth.
Another focus is on agentic AI, systems capable of autonomous decision-making with minimal human oversight. The commission defines these as high-risk tools that require special attention in regulatory frameworks. The guidance would need to address how such systems interact with healthcare workflows and whether they can be safely integrated without compromising patient care.
Training, accountability, and patient rights
Healthcare professionals will need targeted training to use AI tools safely, including understanding their limitations and potential risks. The commission recommends integrating AI education into initial professional training, postgraduate programs, and continuing development. Healthcare providers must also ensure staff are trained specifically for the AI systems they deploy, particularly those with automated decision-making capabilities.
Accountability for AI safety should be shared among manufacturers, providers, clinicians, regulators, and policymakers. The report calls for clear agreements outlining responsibilities, such as cybersecurity measures and staff training. It also addresses liability when AI contributes to patient harm, noting that legal reforms may be necessary to provide fair routes for redress. In the meantime, the health system should ensure AI adoption doesn’t reduce patients’ ability to challenge harmful outcomes.
Patients and the public should have a consistent role in AI regulation, including periodic surveys on attitudes toward healthcare AI. The commission also recommends stronger cybersecurity requirements and clearer guidelines for consumer health apps and wearables, many of which lack standardized oversight.
