Tag Archive for: Software as a Medical Device

SaMD

This post on Software as a Medical Device (SaMD) development is written by Mercedes Massana, the Principal Consultant of MDM Engineering Consultants. 

SaMD software is software intended to be used for one or more medical purposes that do not require embedding in a medical device. These purposes can range anywhere from helping to diagnose or treat disease, helping in the clinical management of a disease, or providing information to help with the clinical management of a disease. SaMD differs from other medical device software in that it will operate on different platforms and will interconnect with other devices, carrying with it an increased cybersecurity risk and a commensurate increase in the use of off-the-shelf software. 

On the surface it may appear that the development of SaMD software is no more difficult than the development of Medical Device embedded software, but appearances can be deceiving, and the development of SaMD products can be quite challenging. In order to deal with these challenges, there are four key best practices that should be followed for SaMD software development.  

These practices are:  

  1. Make use of standards and guidance documents  
  2. Apply the right level of rigor 
  3. Understand the difference between Verification and Validation  
  4. Implement a post-market monitoring program

Best Practice #1– Making Use of Standards and Guidance Documents 

Although standards development organizations and regulatory bodies have only started to scratch the surface in the creation of standards and guidance documents to help SaMD development organizations, there is a sufficiently detailed body of work available to help development organizations succeed. The most relevant of these are the documents generated by the International Medical Device Regulators Forum (IMDRF) related to SaMD and IEC 82304 Safety and Security of Health Software Products. The IEC standard points to other well-known standards, such as IEC 62304 Software Development Lifecycle, IEC 62336 Usability Engineering and ISO 14971. Additionally, several FDA guidance documents are available that apply to all medical device software and are useful for the development of SaMD, these include General Principles of Software Validation, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, Off-The-Shelf Software Use in Medical Devices and the FDA pre and post market cybersecurity guidance, as well as other guidance documents 

Best Practice #2 –Applying the Right Level of Rigor  

Within the development of SaMD, a clear understanding of the scope and intended use of the product is necessary, and to that end, it is necessary to have a method to gauge the risks associated with SaMD use. The IMDRF “Software as a Medical Device” Possible Framework for Risk Categorization and Corresponding Considerations, IEC 62304 , and FDA Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices all provide a method for risk based classification of SaMD. The rigor applied to the development of SaMD should be commensurate with the level of risk. IEC 62304 uses the Safety Classification to define the level of rigor of activities to be performed in the software lifecycle based on the level of risk. Ideally, the development process is sufficiently flexible to avoid the over-engineering or under-engineering of the SaMD in question. A development process that is adaptable requires organizational maturity and experience, in order to perform the right set of activities and understand the value provided by those activities.

RELATED: Regulatory Shift for Machine Learning in Software as a Medical Device (SaMD)

Best Practice #3 – Understanding the Differences Between Verification and Validation  

SaMD is a medical product, a system in and of itself. Therefore, SaMD system requirements must be verified to have been implemented correctly, and  SaMD user’s needs must be validated to ensure that the product satisfies the user’s need and that the right product was built for the customer. Validation typically includes human factors testing, clinical evaluation to determine the efficacy of the software for its intended use, and a demonstration that risk controls are effective. This requires more than just your standard set of software testing, which typically consists of code reviews, unit testing, static analysis, integration testing and requirements-based testing. 

Best Practice #4 – You Are Not Done When the Software is Released  

Your SaMD software has successfully completed verification and validation, has been cleared by regulatory agencies, and is now ready to go live. Time to breathe a sigh of relief and congratulate the team for a job well done. Pop the champagne and throw a party, you have all earned it. Enjoy the festivities, but afterwards turn your attention promptly to monitoring the performance of the SaMD post launch. Rigorous and exhaustive as your testing was, you can never anticipate every possibility. Be ready to respond to identified software defects and emerging cyber threats and deploy patches or software fixes as issues are identified.  The nature of cybersecurity risks is ever-changing, and no system, regardless of how well-designed, or rigorously-tested, is free of software defects. Additionally, customer input can provide opportunities for enhancements and improvement.  However, be aware that proposed changes can impact the intended use of the device, the classification of the SaMD, and can ultimately result in additional regulatory submissions. It is paramount that prospective SaMD developers evaluate changes thoroughly and plan actions related to these changes to avoid surprises that can delay, or outright derail, SaMD implementation. 

In summary, SaMD can provide great benefits to the user; however, to successfully launch SaMD that meets the user’s needs, several best practices should be utilized in their development. These best practices, making use of standards and guidance documents, applying the appropriate level of rigor in the development of SaMD, understanding the difference between Verification and Validation, managing change appropriately, and implementing a good post-market monitoring program.  These best practices are indispensable in ensuring a safe and effective SaMD product. 


Machine Learning in Software as a Medical Device

Machine Learning in Software as a Medical Device

Machine learning (ML), a subset of artificial intelligence, has become integral across software in all industries, and the medical and life science spaces are no exceptions. ML can help medical systems improve the identification and diagnosis of disease, create personalized medicine, and help with drug discovery and manufacturing (just to name a few areas). Current guidance and regulations require validation of the final version of the software prior to a release—so what does this mean for Software as a Medical Device (SaMD) systems that incorporate ML and ‘continual’ algorithms designed to accumulate and improve knowledge after a system’s release in the market?

The Current Situation

The FDA and other regulatory agencies currently lack the guidance needed for medical device manufacturers to include continual algorithms that adjust and improve post-market submissions. A changed algorithm would require a premarket review for each minor adjustment due to the potential impact on patient care.

There are many medical ML products on the market today, which currently use the DeNovo and 510(k) process, but the process is locked. Otherwise, the ML product would violate regulatory control. With this limitation, the FDA has been listening to input from the industry to help inform potential guidance and regulatory change.

Future FDA Regulatory Process

In April 2019, the FDA released a paper called “Proposed Regulatory Framework for Modifications to Artificial Intelligence/ Machine Learning (AI/ML) Based Software as a Medical Device (SaMD).” The paper describes and outlines a possible approach to a premarket review for artificial intelligence and machine learning-driven software modifications.

The proposed framework includes elements from the FDA’s current premarket programs including:

  • IMDRF’s risk categorizing principles
  • FDA’s benefit-risks framework
  • Risk management approaches
  • IEC 62304 principles
  • Change management approaches

One of the key proposed expectations from the FDA would be a commitment from manufacturers to be transparent with the constant monitoring of artificial intelligence and machine-learning-based software providing periodic updates, including any changes to algorithm protocols.

The proposed framework received positive feedback from the industry, and the FDA has now decided to implement the discussion paper with guidance. It is expected to be released sometime in 2021. The intended premarket approval process will allow a trusted manufacturer (see below for definition) to make pre-approved post-release changes only if the manufacturer follows a predetermined change control plan.

What is a Trusted Manufacturer?

So what does the FDA mean by a trusted manufacturer?

Firstly, a trusted manufacturer should ensure they have an efficient and proactive quality system in place. Some of the key quality-related areas to focus on are:

  • Design controls and required documentation to prove to the FDA exactly how the manufacturer has provided for the safety and efficacy of a system or device.
  • Design verification and validation establishing that what the manufacturer has built works for the end user as intended.
  • Risk management, ensuring that any applicable hazards have been identified and that mitigations have been implemented.
  • Iterative design reviews, allowing risks and omissions to be seen quicker, reducing total in-field corrective actions or bugs.

Several other considerations are critical in becoming a ‘trusted manufacturer’ under the anticipated guidance, some of which include:

  • The ability to follow good machine learning practices during all design stages.
  • Ensuring that all algorithm changes that are implemented are done according to pre-specified objectives and any applicable change protocols.
  • Trusted manufacturers are expected to document how the system will learn both pre- and post-release.
  • Ensuring the integrity of reference data used by continual algorithms.

This is an exciting time for medical and life science companies embarking on or continuing their ML product’s journey. Connect with us to find out how Jama can help!