medical device risk management Archives

Tag Archive for: medical device risk management

This post on Software as a Medical Device (SaMD) development is written by Mercedes Massana, the Principal Consultant of MDM Engineering Consultants.

SaMD software is software intended to be used for one or more medical purposes that do not require embedding in a medical device. These purposes can range anywhere from helping to diagnose or treat disease, helping in the clinical management of a disease, or providing information to help with the clinical management of a disease. SaMD differs from other medical device software in that it will operate on different platforms and will interconnect with other devices, carrying with it an increased cybersecurity risk and a commensurate increase in the use of off-the-shelf software.

On the surface it may appear that the development of SaMD software is no more difficult than the development of Medical Device embedded software, but appearances can be deceiving, and the development of SaMD products can be quite challenging. In order to deal with these challenges, there are four key best practices that should be followed for SaMD software development.

These practices are:

Make use of standards and guidance documents
Apply the right level of rigor
Understand the difference between Verification and Validation
Implement a post-market monitoring program

Best Practice #1– Making Use of Standards and Guidance Documents

Although standards development organizations and regulatory bodies have only started to scratch the surface in the creation of standards and guidance documents to help SaMD development organizations, there is a sufficiently detailed body of work available to help development organizations succeed. The most relevant of these are the documents generated by the International Medical Device Regulators Forum (IMDRF) related to SaMD and IEC 82304 Safety and Security of Health Software Products. The IEC standard points to other well-known standards, such as IEC 62304 Software Development Lifecycle, IEC 62336 Usability Engineering and ISO 14971. Additionally, several FDA guidance documents are available that apply to all medical device software and are useful for the development of SaMD, these include General Principles of Software Validation, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, Off-The-Shelf Software Use in Medical Devices and the FDA pre and post market cybersecurity guidance, as well as other guidance documents

Best Practice #2 –Applying the Right Level of Rigor

Within the development of SaMD, a clear understanding of the scope and intended use of the product is necessary, and to that end, it is necessary to have a method to gauge the risks associated with SaMD use. The IMDRF “Software as a Medical Device” Possible Framework for Risk Categorization and Corresponding Considerations, IEC 62304 , and FDA Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices all provide a method for risk based classification of SaMD. The rigor applied to the development of SaMD should be commensurate with the level of risk. IEC 62304 uses the Safety Classification to define the level of rigor of activities to be performed in the software lifecycle based on the level of risk. Ideally, the development process is sufficiently flexible to avoid the over-engineering or under-engineering of the SaMD in question. A development process that is adaptable requires organizational maturity and experience, in order to perform the right set of activities and understand the value provided by those activities.

RELATED: Regulatory Shift for Machine Learning in Software as a Medical Device (SaMD)

Best Practice #3 – Understanding the Differences Between Verification and Validation

SaMD is a medical product, a system in and of itself. Therefore, SaMD system requirements must be verified to have been implemented correctly, and SaMD user’s needs must be validated to ensure that the product satisfies the user’s need and that the right product was built for the customer. Validation typically includes human factors testing, clinical evaluation to determine the efficacy of the software for its intended use, and a demonstration that risk controls are effective. This requires more than just your standard set of software testing, which typically consists of code reviews, unit testing, static analysis, integration testing and requirements-based testing.

Best Practice #4 – You Are Not Done When the Software is Released

Your SaMD software has successfully completed verification and validation, has been cleared by regulatory agencies, and is now ready to go live. Time to breathe a sigh of relief and congratulate the team for a job well done. Pop the champagne and throw a party, you have all earned it. Enjoy the festivities, but afterwards turn your attention promptly to monitoring the performance of the SaMD post launch. Rigorous and exhaustive as your testing was, you can never anticipate every possibility. Be ready to respond to identified software defects and emerging cyber threats and deploy patches or software fixes as issues are identified. The nature of cybersecurity risks is ever-changing, and no system, regardless of how well-designed, or rigorously-tested, is free of software defects. Additionally, customer input can provide opportunities for enhancements and improvement. However, be aware that proposed changes can impact the intended use of the device, the classification of the SaMD, and can ultimately result in additional regulatory submissions. It is paramount that prospective SaMD developers evaluate changes thoroughly and plan actions related to these changes to avoid surprises that can delay, or outright derail, SaMD implementation.

In summary, SaMD can provide great benefits to the user; however, to successfully launch SaMD that meets the user’s needs, several best practices should be utilized in their development. These best practices, making use of standards and guidance documents, applying the appropriate level of rigor in the development of SaMD, understanding the difference between Verification and Validation, managing change appropriately, and implementing a good post-market monitoring program. These best practices are indispensable in ensuring a safe and effective SaMD product.

With the rapid innovation in digital health such as clinical web and mobile software applications which do not traditionally classify as a “device” due to its lack of custom electromechanics, the Software as a Medical Device guidance (SaMD) was adopted by the FDA along with global regulatory leaders to ensure such software products adopt the same general medical device principles to demonstrate safety, effectivity and performance. The creation of SaMD was heavily influenced by new entrants unfamiliar with medical device regulations and terminology developing a broad spectrum of applications in order to support the new and growing industry.

On August 25th, 2020 Jama Software’s Clay Moore moderated a webinar featuring Beanstock Venture’s CEO, Shawnnah Monterrey, and past and present members of the FDA including John Murray and Bakul Patel.

Watch the full webinar now or read an abbreviated version of the transcript below.

My Software Is a Medical Device (SaMD). Now What?

Isabella Kennedy: As an industry, we are starting to see an influx of software applications being used for clinical support to inform clinical decisions, diagnosis, and treatment. We thought with all the new entrants unfamiliar with medical device regulations, it might be helpful to provide an overview and answer some questions. We’ve put together this amazing team of panelists that come from different organizations with tremendous offerings.

For example, BeanStock Ventures provides Software Online Agile Regulatory training, otherwise known SOAR, QMS, DHF audit and assessments, as well as medical device software development. Software CPR provides expert regulatory and standards compliance advice, FDA negotiations and strategies, and internationally recognized training courses.

We are all, of course, familiar with the FDA that provides numerous guidance documents related to software and specifically, in this case, SaMD. Jama provides software lifecycle management tools to aid in automating the process in facilitating compliance. As a follow-on, we will be sharing helpful resources from each company that might be available with you.

Let’s just have the panel members introduce themselves, starting with Shawnnah Monterrey.

Shawnnah Monterrey: Hi, I’m Shawnnah Monterrey, CEO of BeanStock Ventures.

Clay Moore: This entire category is around the definition and classification that drives regulatory status.

As technology evolves, and software becomes a platform and location independent, it is more important to understand what it does rather than where it resides. How does one determine if their software product is a medical device? Can you describe some elements of an intended use statement?

John Murray: Well, I think that everyone needs to understand that you can describe a device or a product in multiple, multiple ways. You can have engineering description, architectural description, all this stuff. But intended use is kind of special. It’s a special subset of all of that.

The focus of that description or that understanding, you need to focus on the patient and the practitioner. I think that’s the very important thing. I’ve seen lots of descriptions that don’t talk anything about the patient or the clinical use. When you’re addressing intended use, you may get confused by the definition and the regulation.

But you need to start asking yourself basic questions. Some of the questions I’ve come up with are, “So, I’m a doctor. How is this product going to make me or help me practice medicine better? How is this product going to help my patients live better lives?” Those focuses, clinical focused questions are very important to be honest about when you’re addressing that intended use statement.

That’s my first comment on that. Number two is that you have multiple customers here. You have the engineering staff, RAQA, marketing, and clinical staff. My suggestion before you tackle the intended use statement is to take a step back, and write a software product description, and get input from all four of those groups.

You need to balance the wheel and you’ll end up with a document that basically everyone can read, everyone understands. That will become the first step to risk management, the first step to design, the first step to regulatory status. It’s well worth the money and the time to do this correctly. The third caution I have about intended use is it’s never static. You start out today with an intended use and everything is well under control.

You design a great product, you have great customers, everything is fine, but before you know it, it’s being used in a different way. Your marketing team starts selling it in a different way. You need to make sure you keep that dynamic and alignment correctly. You don’t want to be in a situation where your intended use is stated this way, but your marketing intended use is a different way.

There are three things. One is to lean towards the clinical and patient. Write a full software product description. Number three; keep track of the dynamic because every SaMD that I’ve ever seen is always sliding into the future. It never stays the same because once you have it, you go, “I can use it for this now. This is a great opportunity so let’s move on.”

Clay Moore: That’s fantastic. I think we’ll spend some more time later talking about the iterative nature of SaMD products. Bakul, what’s some additional thoughts that you might have around intended uses?

Bakul Patel: Yeah, I think John hit up on most of it. I will just add to the concept of the clinical views and who the user is. From IMDR, and through work that we’ve done in defining and categorizing what Software as a Medical Device should look like and what’s a clear that definition looks like, I think we spend a lot of time talking about that.

Internationally, we thought about two things. What the software is doing. Where it’s going to be used. Who’s going to use it. How it’s going to be used. If you think about what the software is doing in the care, in the practice and then you have the other dimension of the context of sort of where it’s going to be used. Context is a bigger situation.

The term that we use in the IMDR framework is the healthcare situation and condition, which means it considers the disease, the condition of the disease through a varying spectrum of the disease you are trying to target. It’s not everything and anything. You can’t just say heart disease and cover the entire gamut because there are many things that you need to consider.

Even in there, there is a spectrum. Then you have the user who is going to use it in certain areas and certain types. Just working through the logic of where it’s going to be used, who’s going to use, and how it’s expected to be used. It’s sort of honing down on your intentions. I like the point John made about it’s not static. I’m going to give you a non-static version of what John was talking about.

Not just about static, about QMS. Start thinking you want to do this but then you find out your technology can only do this. There’s a difference between where you started and where you ended up. The result and what you want to put out to market should be exactly to what your product is performing. I think when that alignment happens between what the product does versus what you want the product to do is really what the regulatory terms calls intended use.

Clay Moore: Thanks for that additional context there. Shawnnah, what are some other thoughts you might have around intended use? Also, if you could start to peel back a little bit and talk about what if there are gray areas and we’re not sure about whether this really should be classified as SaMD?

Shawnnah Monterrey: Sure. I like to break down the intended use into four pieces. I like to focus on the user, the end user of the clinical product, the clinical utility of the product, the environment in which it’s going to be used, which could vary, and also the demographics of the patient.

Giving you an example, maybe I can walk through a simple example of maybe a heart monitor. A heart monitor, the environment would be, for this particular product, would be home use. The end user for that would actually be the patient. The demographic of the patient let’s say it’s age, I don’t know, 55 and up with heart condition that can be self-monitored.

The clinical utility is to monitor heart arrhythmia. Obviously, you would start there, and you can extrapolate further but I like to divide it in those areas to make sure all aspects of the intended use are covered. In terms of an example, I can give one particular example. I’m not sure if it’s still valid but if you consider Fitbit as traditionally not a medical device.

I think they’re moving towards getting it cleared or they may have cleared a version of it as a medical device in recent times. There was a lot of clinical applications being built on top of Fitbit. How do you determine if your product is a clinical device at that point if you’re taking data from the Fitbit and displaying it to an end user?

If you’re just taking data that’s being provided from the actual tool itself and you’re just passing it through, reading it basically, and rendering it, showing it on the display, generally, that’s not necessarily considered a clinical device. But if you take that data and you start providing additional analytics on top of it or start making clinical inferences based on that data, that’s the point where start getting into the medical device space.

Clay Moore: That’s great. What if the software helps the patient to not be in a clinical setting, if it’s used at home? John, maybe you’ve got some comments on the, again, would we think of this as something that’s being classified as SaMD or not?

John Murray: I don’t think that the device, not a device question starts with at home use or in clinic use. I think that’s a secondary question. It may determine the actual classification of the product. The product may be for monitoring blood glucose, take an example. You could have one product that’s for design to be used in a clinic, one for use at home. They may have different risk determinations.

That may make the classification different. It always must start with the specific intended clinical use, in my opinion. Part B would be now what’s the risk of this application? That varies depending upon the patient population, the environment of use, and a bunch of other questions that could come into being. That would be how I would start framing that up as a question.

Clay Moore: That’s great. Bakul, any other commentary on example of things that are SaMD or aren’t SaMD? Maybe some gray areas.

Bakul Patel: Yeah. Recently, in the act or jurisdiction doesn’t extend to things that are not meant for clinical purpose. It must be for caring, mitigation, treatment or preventing a disease. That’s statutory definition.

An activity tracker just measuring how many steps you’re taking per day and helping you stay healthy is not something we would consider as a medical device even from that perspective. Then certainly, I think this is what Shawnnah was talking about, is you take that steps and say because you took X, or you took less or took more steps, it means certain disease, or it means certain condition.

Clay Moore: Great. I think each of you mentioned risk at some point in time during each of your responses. Let’s move into more the things that are related to risk and the amount of effort that’s behind these things. Once a company determines their software product is a medical device, the next step is to determine its risk category.

Risk categories are used to determine the rigor required to ensure that the product is safe, is effective, and it needs to perform its criteria necessary. For SaMD, risk is classified using a framework where one is the lowest risk and four is the highest risk. John, can you discuss why understanding risks is important when developing and maintaining a software to medical device?

John Murray: Well, the fundamental basic idea is you don’t want anything bad to happen to good people. That’s the purpose of all these rules. We want to make sure your product doesn’t hurt anybody in a way that’s preventable. That’s the simple explanation of that. I want to talk a little bit about one of the special elements of SaMD in my opinion.

That is that we have a specific risk category that I often see with many of my clients gets left off. I’m going to call that engineering or environmental risk, which is a little bit different than clinical risk, traditional clinical risk. Because you know you’re using a SaMD, you need to think about all the elements of that platform choice that may have an impact on harming people.

It may have an impact on risk. The beauty of this is that once you set up this model, you can reuse these questions repeatedly with every product. One simple example is what if the Bluetooth connection becomes lost? What kind of impact is that going to have on my patient? What kind of impact is that going to have on my delivery of my intended use?

All the known features of a general-purpose computing platform, wi-fi, Bluetooth, all that kind of stuff, you can ask those questions. The beauty of this is that I know the engineers are asking these questions all the time. Those questions never get translated over to the regulatory side where the risk management thing is set up.

The idea is that you want to do what you can do. Maybe if you lose Bluetooth for five seconds, it’s not a big deal but what if you lose it for an hour or two hours? There’s a huge number of SaMD applications where you can lose that connection for a short period of time. It’s not going to be an impact. Do you need to know when that happens, and do you need to inform the patient when that happens? In some applications, a patient needs to know they’re not connected.

One of the other ones that I thought of and we saw in some recalls was that what if you start your algorithm, and you are resource deprived? You don’t have the right amount of memory.

You don’t have the right amount of algorithm power, computing power. All these questions, I know they get asked all the time by the engineering staff. The engineering staff needs to be accustomed to contributing that information to the design and to the regulatory affairs side so that when you present your case, you’ll say, “Yeah, we covered all the environmental engineering risk and here’s our list here.”

That gets sandwiched together with the clinical risk, which I’m not a doctor so I don’t like to talk about that very much. That’s a hugely important thing, is being honest. Being honest that bad things can happen, but will they have an impact on my clinical application?

Clay Moore: Shawnnah, why don’t we go over to you and would love to get some additional color on that as well as maybe thinking about what are some aspects that influence safety, effectivity, or performance?

Shawnnah Monterrey: When I look at performance, I think about technological performance and analytical performance. From a technical perspective, especially for a SaMD product where it can be hosted in numerous environments, you really want to go back and understand how the operator is going to be using your system and the impact that that environment may have on your product.

I’ll give you an example of something that could impact performance for a hosted cloud application could be the latency of the network. Something you can’t control, so it’s important to really understand the environment in which your product is going to be used and then how that would then influence some of the safety associated with some hazards to be identified.

A top four list can include stuff like no result. What would happen if the network goes down and the end user wasn’t able to receive results? Delayed results as a result of network latency and is there any impact to false positive or false negative as a result? I think network latency may not directly influence a false positive and false negative but results in a no result or a delayed result.

That’s something that you should assess and determine the severity based on, again, the intended use of your product. From another performance perspective would be the analytical side. That’s focused more on the sensitivity and specificity of your product and really, looking at a false positive and false negatives.

That requires, again, an understanding of your demographics. What is that patient population that is needed to be able to truly test your product to show and prove the analytical performance of the system?

Clay Moore: There was a question around the differences and overlap between SaMD when it’s a mobile medical application and software is inside a medical device considering the same software can be leveraged across different user contexts.

Bakul Patel: I think the intended use of a piece of software does not rely specifically on the platform that it’s residing on. I think that’s one thing you have be very clear about.

The software is going to take advantage of the computing resources and the surroundings of that environment is what I call it. I don’t think it matters from that perspective. Whenever somebody makes a choice to put a product out, they intentionally market it or design it for a particular product or platform.

When you choose that, I think you need to make sure that you know all the things that you are going to depend on or not depend on in that platform. It could be a multi-thread application or a multi-thread operating environment, but you may not use multi-thread for all kinds of various reasons.

You may know using multi-threading applications may lead to some issue that may raise conditions and you may want to think about that. When you start using operating systems and you are using instructors and you have multi-course going on today, you are going to have to say that, “Even if I used a simple programming language to be on a certain platform, the platform may use my program separately because you have different threads running.”

Just knowing that will help you start answering some of the questions about what do you have to mitigate? How to reduce the risk to a minimum. How do you consider that? I think cloud services is very similar. I think you have very different challenges. Mobile is the same. Very different challenges and different sort of advantages. You just take those into account.

To see more information specific to the medical device development industry, we’ve compiled a handy list of valuable resources for you!

SEE MORE RESOURCES

Building and then bringing a medical device to market as quickly as possible—all while preserving acceptable levels of quality and regulatory compliance—requires adept medical device risk management. By minimizing potential risks such as mislabeling and software-related issues, medical device manufacturers make each product safer for the patients who will use them. All of these risks and others are present through the product development lifecycle, where they must be addressed through specific risk management activities

The ISO 14971 standard, as encapsulated in ISO 14971:2007 and then revised in ISO 14971:2019, is the modern framework for such efforts. An FDA Recognized Consensus Standard, it has a nine-part structure defining the criteria for medical device risk management during production and post-production. ISO 14971 is also required by higher-level regulation under ISO 13485. All medical device companies follow ISO 14971, but their individual approaches to the risk management standard will vary based on not only product type but the actual tools used for handling risk analysis and control measures as well.

Risks as requirements: What’s the best approach to medical device risk management?

Effective medical device risk management is integral to patient health and safety. A study published by The British Medical Journal found that one in 20 patients experiences preventable harm when receiving medical care. Moreover, many medical devices in active use are many years old, meaning that even flaws implemented long ago can continue to pose risks to patients. That means risk must be curbed at every stage of the product lifecycle.

Another way to look at it: Risks are central requirements when it comes to the medical product development process, and safely managing them in accordance with ISO 14971 requires a comprehensive modern solution capable of delivering the necessary coverage, speed and preparation. Risk management is requirements management in the medical device industry. Accordingly, it’s crucial to have the capacity to, for instance, connect eventual verification tests back to requirements, so that teams can be confident of adequate risk mitigation.

However, many existing workflows and tools cannot consistently ensure acceptable compliance, leading to the possibility of recalls, or inadequate workarounds such as alterations to the label or instructions-for-use. Spreadsheets exemplify the limitations of older approaches to requirements management and risk management during the medical device lifecycle.

The problem with document-based processes

Medical device manufacturers may rely on Microsoft Excel to capture risk data and fuel their risk management planning and reporting activities. The potential problems with this approach include:

Limited scalability to teams working at multiple locations.
Siloed data sources that take time to comb through and reconcile.
Human factors such as miskeyed entries or inadvertent deletions.
Difficulty proving compliance, due to lack of end-to-end traceability.

Taken together, these issues make it onerous to maintain and execute on a medical device risk management plan that fulfills all provisions of ISO 14971. This standard requires a combination of risk analysis, evaluation and control – all processes that a risk management plan helps simplify by documenting all of the potential risks across the product lifecycle.

How to more reliably put a risk management plan into action

The risk management planning process should produce a plan that contains product-specific data and follows all standard operating procedures in the domain. The plan should also be a living document that can be continually updated as requirements and risks evolve, as it will serve as the blueprint for ongoing risk management activities such as reporting on hazards and risk control measures and also linking back to requirements. Ensuring acceptable levels of detail and accuracy in the risk management plan is much easier with an all-in-one solution than with a collection of discrete documents.

Let’s say a hypothetical medical device company was developing an MRI machine. If it were centering its risk management processes within a massive Excel sheet, lots of valuable time would be lost to stakeholders on the development team having to constantly review the requirements in the shared asset.

Plus, this highly manual, error-prone process can itself create further complications for overall medical device risk management, such as a risk going initially overlooked due to outdated data in a spreadsheet cell. Going back later to write a report about the risks is not a great alternative to building in risk management throughout the development process—but the right tools are needed for the latter strategy.

By switching to a more modern solution, this development team could instead:

Take advantage of risk plan templates to ramp up more quickly.
See live risk mitigation data, not outdated entries.
Avoid the various administrative risks of Excel, like splitting/merging cells.
Easily adjust probabilities and severities of the defined risks.
Enable real-time collaboration between teams.
Visualize and trace risks across the whole product development lifecycle.
Prove ISO 14971 and other regulatory compliance more easily.

At the end of the development process, the team making this hypothetical MRI machine would be able to see clearly how its verification tests traced back to the risks and requirements it initially set. More specifically, they would know if the product could move the right amount of air, survive the expected transport and storage conditions and comply with all applicable rules and regulations.

Demonstrating ISO 14971 compliance with Jama Connect

Jama Connect offers a modern alternative to document-oriented processes for medical device risk management. Jama Connect is built to help streamline compliance with ISO 14971.

For example, Clause 7 of ISO 14971 requires attention to residual risks, or those risks that exist even after all risk control measures have been implemented. In Jama Connect, those measures can be efficiently defined and linked to corresponding risks for maximum traceability. That way, teams can spot potentially unacceptable risks early on in development and mitigate them before the associated costs and logistics become impractical.

There are many other features within Jama Connect for complying with all clauses of ISO 14971 and modernizing your general approach to medical device risk management to keep up with changes such as the FDA’s Safety and Performance Based Pathway. To learn more, connect with an expert today.

To learn more on the topic of risk management, we’ve compiled some helpful resources for you.

LEARN MORE