Although the U.S. Food and Drug Administration (FDA) doesn’t currently have unique approval processes for medical devices containing artificial intelligence (AI), the technological landscape is rapidly evolving, requiring manufacturers to remain aware and responsive.

Proposing Regulations for Predetermined Change Control Plans

In August 2024, FDA regulators proposed regulating predetermined change control plans (PCCP) and that any relevant decisions would affect AI devices. This news came after previous work completed in 2021, during which FDA officials worked with regulators from Canada and the U.K. to develop 10 guiding principles to shape the development of machine learning technologies. Those involved focused on safety, effectiveness, and applicability to real-life needs. They also believed these guidelines could improve how some devices perform.

Five created principles affect PCCP documentation and the devices requiring it. One that’s highly applicable to machine learning models requires manufacturers to monitor how their deployed models perform and minimize the risks associated with retraining machine learning algorithms. The 2021 guidelines also stipulated that significant changes to AI-containing medical devices may need regulatory oversight, including an extended premarket review process.

Those who developed the guidelines also recognized that in-depth reviews of AI medical devices don’t always match the fast pace of product development. However, when FDA regulators issued their August 2024 draft guidance, they stipulated what future PCCPs for AI devices might require.

Their recommendations advised that a device’s PCCP should detail planned modifications, including manufacturers’ methodology for development, validation, and implementation. Additionally, the document should include the device maker’s assessment of how those changes could impact the product.

FDA authorities envisioned that the recommendations would affect devices subject to the organization’s premarket approval, De Novo, and 510(k) pathways. The agency accepted comments on the proposed changes through November 20, 2024.

As of early December, there were no further concrete details about the potential impacts on manufacturers. However, device makers should remain aware of these developments and plan how to respond to possible outcomes.

Ensuring Medical Devices Follow Cybersecurity Best Practices

The FDA engages in many activities to ensure medical devices are free from excessive cybersecurity risks. That’s particularly important due to the many potential entry points for malicious parties and the potential damage once they’re inside a network or if they’re able to tamper with devices.

One science and technology company created a machine learning-driven innovation that was 96% accurate in predicting the onset of kidney failure with 24 hours’ notice, showing the potential of medical professionals using AI devices to improve patient outcomes. Since the product can be integrated with an electronic health records (EHR) system, developers likely had to implement numerous cybersecurity safeguards.

The FDA also publishes guidance for manufacturers about how to design and maintain their products for cybersecurity. The agency encourages manufacturers to monitor for and disclose vulnerabilities in their devices and describe the proposed solutions for them. When regulators believe the identified issues pose a risk, they may issue a safety communication document that contains relevant details and recommended actions for patients, providers, and manufacturers.

Device makers can reduce the likelihood of future problems by performing comprehensive product audits to find potential issues cybercriminals may exploit. They can then investigate how to minimize or eliminate them to make the products safer for the patients and providers who use them.

Establishing an AI Safety Program

In October 2023, the Biden-Harris administration issued an AI safety-related executive order. It has broad aims but some of the order’s components relate to medical devices. The FDA is within the Department of Health and Human Services (HHS). The order said that the HHS would establish a reporting program for reporting and remedying harms and unsafe practices related to artificial intelligence in healthcare.

Some researchers argue there is no better time to create one. In one case, a team pulled records from the FDA’s Manufacturer and User Facility Device Experience (MAUDE) database, which contains mandatory and voluntary reports linked to medical devices. The group sought to determine the number potentially related to AI products.

They found and analyzed 429 reports, determining that more than 25%, or 108 entries, were possibly related to AI medical products. They concluded that by searching for specific phrases and keywords commonly mentioned when describing AI medical technology. However, in 34.5% of cases, the report’s language did not contain enough information to connect an incident to an AI product.

The researchers asserted that patient safety reports by themselves are insufficient for identifying safety issues with AI products or whether AI contributed to an error. They clarified that one reason is those making these submissions may not know how AI works behind the scenes in a product, making them unaware of when or if it contributes to a problem.

Users may become particularly excited about how AI medical devices could improve their lives but be unaware of potential downsides, a sentiment that also exists for products without AI features. In one case, a woman pulled 916 heart readings from her smartwatch in a year. Unfortunately, her excessive vigilance led to a new health anxiety diagnosis.

Requiring More Clinical Validation Data

In addition to researchers who say the FDA’s current safety reporting programs fail to effectively flag AI risks, some believe AI products approved by the agency should meet a standard for sufficient clinical validation data.

A device is clinically validated when test results show it’s accurate enough to meet clinical standards. However, when one group of researchers analyzed more than 500 FDA-authorized AI medical devices, they found many lacked clinical validation data. Their investigation indicated that was true for approximately half the products examined.

One of the researchers involved with this product emphasized that although AI device manufacturers can boost credibility by getting their products authorized by the FDA, that clearance doesn’t mean they all have the necessary clinical validation enabled by actual patient data.

The research group also scrutinized 521 FDA-authorized AI medical devices, finding that 226 had no clinical validation data. In some cases, the devices relied on computer-generated images rather than images of real patients, which didn’t technically meet clinical validation standards.

Additionally, the researchers said the FDA’s current draft guidance about clinical validation doesn’t clearly distinguish between the three types of clinical validation studies manufacturers can do. They recommended that regulators provide more clarity on the matter and later met with the agency’s directors to share their findings.

It’s too early to say whether the agency will update its documentation and practices based on this study. However, these conclusions, like others mentioned earlier, indicate that the FDA needs to do more to ensure AI medical devices are safe and effective for all who rely on them.

Tracking a Developing Situation

Artificial intelligence in medical devices is becoming much more prominent but is still relatively new. FDA regulators must continue monitoring all relevant aspects, including the shortcomings noted here and elsewhere.

AI has pushed many industries into new territory. As it continues disrupting healthcare, regulators must refrain from prioritizing innovation while sacrificing patient safety. Medical device makers must remain aware of new developments and plan accordingly to give their products the best chances of getting FDA approval and earning user confidence.

More from this author: What is Nitinol and How is it Leveraged in Medical Device Design?

Dive deeper into AI in medtech with this feature article by managing editor Mike Barbella

Emily Newton is the editor-in-chief of Revolutionized. She’s always excited to learn how the latest industry trends will improve the world. She has over five years of experience covering stories in the science and tech sectors.