Risk Management

Expert Perspectives: A Method to Assess Benefit-Risk More Objectively for Healthcare Applications

By |

In this blog, we’ll recap a section of our recent Expert Perspectives video, “A Method to Asses Benefit-Risk More Objectively for Healthcare Applications” – Click HERE to watch it in it entirety.

Expert Perspectives: A Method to Assess Benefit-Risk More Objectively for Healthcare Applications

Welcome to our Expert Perspectives Series, where we showcase insights from leading experts in complex product, systems, and software development. Covering industries from medical devices to aerospace and defense, we feature thought leaders who are shaping the future of their fields.

In the complex world of healthcare, evaluating benefit-risk is crucial to successful product development and patient outcomes. Our expert perspectives video, “A Method to Assess Benefit-Risk More Objectively for Healthcare Applications,” offers actionable insights for healthcare innovators aiming to meet rigorous regulatory requirements while ensuring patient safety and efficacy.

In this episode of Expert Perspectives, Richard Matt breaks down a streamlined, objective method for benefit-risk analysis. He explores a structured frameworks and data-driven approach that help teams make balanced decisions, mitigate risks early, and stay compliant with regulatory standards, including FDA and ISO guidelines.

This patent-pending approach helps organizations navigate challenges, foster innovation, and ultimately bring safer, more effective healthcare solutions to market.

Below is a preview of our interview. Click HERE to watch it in its entirety.

Kenzie Jonsson: Welcome to our expert perspective series where we showcase insights from leading experts in complex product, systems, and software development. Covering industries from medical devices to aerospace and defense, we feature thought leaders who are shaping the future in their fields. I’m Kenzie, your host, and today, I’m excited to welcome Richard Matt. Formerly educated in mechanical, electrical, and software engineering and mathematics, Richard has more than thirty years of experience in product development and product remediation. Richard has worked with everyone from Honeywell to Pfizer and is now a renowned risk management consultant. Today, Richard will be speaking with us about his patent pending method to assess benefit-risk more objectively in health care. Without further ado, I’d like to welcome Richard Matt.

Richard Matt: Hello. My name is Richard Matt, and I’m delighted to be speaking with you about our general solution to the problem of assessing whether the benefit of a medical action will outweigh its risk. I’ll start my presentation by saying a few words about my background and how this background led to the benefit-risk method you’ll be seeing in the presentation.

To understand my background, it really helps to go back to the first job I got out of undergraduate school. I graduated with a degree in mechanical engineering and an emphasis in fluid flow. And my first job was in the aerospace industry at Arnold Engineering Development Center, at a wind tunnel that Baron von Braun designed. I worked there as a project manager, coordinating various departments with the needs of a client who brought models to be tested. These are pictures of the ADC’s transonic wind tunnel with its twenty-foot by forty-foot long test section that consumes over a quarter million horsepower when running flat out. Those dots in the walls are holes, and a slight suction would pull the out on the outside of the wall to suck the air’s boundary layer through the holes. So a flight vehicle appeared more closely to match its flight air characteristics in free air. It was amazing place to work.

We could talk about aerodynamic issues and thermodynamic issues like why nitrogen condenses out of the air at mach speeds above six or why every jet fighter in every country’s air force has a maximum speed of about mach three and a half. But to stay on the topic of benefit-risk, the reason or my intro to this, the reason I was brought this up was that I saw here firsthand the long looping iterations that came from different technical specialties, each approaching the same problem from the respective of their technical specialty. I found it very frustrating and the, following analogy very apt, after getting, so each of our technical specialties would look at the same problem, the elephant from their own view. And I found myself getting frustrated with my electrical and software engineering coworkers, that they didn’t understand what I was talking about, but I knew realized soon I didn’t understand what they were talking about either.

So I decided I wanted to become part of the solution to that problem by going back to graduate school and getting myself rounded out and my education so I could talk to these folks from their perspective also. So I went back to grad after mechanical and undergraduate, went back to graduate school in electrical and mathematics and picked up enough software. I started teaching, programming also in college. I developed there a solution for the robot arms in those wind tunnels to to control a robot arm for every possible one, two, or three rotational degree of freedom arm, and that was my graduate thesis. After I completed my thesis, I felt empowered to start, my work doing going wherever I wanted doing whatever I wanted to do and realized that if I wanted to do anything significant, it would take many years, and I decided to focus on teamwork. Does that sound pretty good?

RELATED: Jama Connect® for Medical Device & Life Sciences Development Datasheet

Matt: My ability to work across technical boundaries enabled me to bring exceptional products to the market. For instance, I brought an Internet of Thing  (IoT) device to the market during the nineteen nineties before Internet of Things was a thing. And I rapidly advanced while I was working as a VP of engineering at a boutique design firm in the Silicon Valley. These are a few of the clients that I had, through the work that I’ve done over the years.

And, the combination of the breadth of my formal training and my system perspective for solving problems has really helped me work across continue to work across boundaries, so that I’ve worked for companies to help them establish their pro product requirements, trace requirements, do V and V work. I’ve done a lot of post-market surveillance work. I established internal audit programs. I’ve been the lead auditee when my firm is audited. Done had significant success accelerating product development and has been on work on. So mixed in with all of these works, I special I started specializing into risk management as consulting focus versus something I just did normally during development.

And since the defense of a patent requires notice, I’ll mention that the material here is being pursued on the patent, and, would like to talk with anyone who finds this interesting to pursue after you’ve learned about it. So let me start my presentation on benefit risk analysis by talking about how important it is to all branches of medicine and the many problems we have implementing it. The solution I’m gonna come up with, I’ll just outline here briefly so you can follow as we’re going through the presentation. I’m gonna first establish a single and much more objective metric to measure benefit and risk than people traditionally use. I’ll be accumulating overall benefit and risk with sets of metric values from this first metric. And finally, we’ll show how to draw a conclusion from the overall benefits and risk measurements of which is bigger benefit or risk.

So in terms of importance, historically, benefit-risk has been with medicine for millennia. It’s a basic tenant to all of medicine. The first do no harm goes all the way back to the quarter of Hammurabi 2,000 BC, and it legally required physicians to think not just about how they can help patients with treatment or what harm they might cause to treatment and making sure that the balance of those two favor the patient is very much the benefit-risk balance that we look at today. The result we’re gonna talk about is gonna be used everywhere throughout medicine with devices, with drugs, with biologics, even with clinical trials.

So is that fundamental cross medicine? How it’s used currently?

If you are in one of the ways developing new products, benefit-risk determinations have to be used in clinical trials to show that they’re ethical to perform, that we’re not putting people in danger needlessly. Benefit-risk determinations are the final gate before a new product is released for use to patients. And I have a quote here from a paper put out by AstraZeneca saying the benefit-risk determination is the Apex deliverable of any r and d organization. There’s a lot of truth to that. It’s the final thing that’s being put together to justify a product’s release. And so it has a very important role here for FDA and has a very important role for pretty much the regulatory structure of every country, including the EU.

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Medical Device & Life Sciences

Matt: In terms of creating a quality system, every medical company is required to have one. Benefit-risk determinations are used to assess a company’s quality system. This is per the FDA notice about factors on benefit-risk analysis. When regulators are evaluating company’s quality system, they’ll use benefit-risk to determine if nothing should be done, if a product should be redesigned, if they should take legal actions against a company of a range of possibilities from replacing things in the field to stopping products from being shipped. It’s also a key in favorite target for product liability lawsuits, because of how subjective it is, and we’ll get to that in a moment. It can also be used for legal actions against officers. So benefit risk is a really foundational concept for getting products out and keeping products out and keeping companies running well. Just a bit of historical perspective of medical documentation and development. We have here, I cited four different provisions of the laws, regarding medical devices in the United States. This is a small sampling.

The point I’m trying to make here is that each of these summaries of the laws discuss continually evolving, continually growing, more rigorous standards for evidence, more detailed requests for information from the regulators to the instrumentation development companies to the product development companies. So first, medical products are heavily regulated. We have the trend of increasing analysis and rigor. Per ISO 142471, and this is an application standard that is highly respected in the medical device field. A decision as to whether risks are outweighed with benefits is essentially a matter of judgment by experienced and knowledgeable individuals.

And this is our current state of the art.

Not that everybody does it this way, but this is the most common method of performing benefit-risk analysis. And benefit-risk analysis by this method, has a lot of problems because it’s based on the judgment and it’s based on individuals, and both of those can change with different settings. That’s why it’s a favorite point of attack for product liability lawsuits.

This quote was true in 1976, when medical devices were put under FDA regulation, but significantly remains unchanged nearly fifty years laters. Benefit-risk determinations are an aberration and that unlike the rest of medicine, they have not improved over time. They’ve remained a judgment by a group of individuals. In, twenty eighteen, FDA was, approached by congress to set a goal for itself of increasing the clarity, transparency, and consistency of benefit risk assessments from the FDA.

This was in human drug review as the subject, and the issue was that various drug companies had gotten very frustrated with the FDA for disagreeing with their assessments of what benefit-risk should look like. And to repeat again, when you have a group of individuals making a judgment, that’s gonna lead to inconsistencies because both the group and their own individual judgment will vary from one situation to the next. I have another, quote here from the article from AstraZeneca. The field of formal and structured benefit-risk assessments is relatively new.

RELATED: Application of Risk Analysis Techniques in Jama Connect to Satisfy ISO 14971

Matt: Over the last twenty years, there’s still a lack of consistent operating detail in terms of best practice by sponsors and health authorities. So this is an understatement, but a true statement. We have had a lot of increasing effort over the last few years because if people are dissatisfied with the state of benefit-risk assessments, they want to do better than this judgment approach. And so there have been a plethora of new methods developed. I’ve found one survey here that summarize fifty different methods just to give you an idea of how many attempts there are. And I went through those fifty methods.

The other thing that’s interesting to see is the FDA’s attempt to clarify benefit-risk assessments. I have here five guidance documents from the FTA, and I would put forth the proposition that anytime you need five temps five attempts to explain something, it means you didn’t understand the thing well in the first place or failing about a bit trying to get it done right. I think this is also held up by the drug companies, pressure on congress to get FDA to improve their clarity and consistency of benefit-risk assessments.

So here’s the, fifty methods that I found in one study of benefit-risk assessments. They have them grouped into, a framework, metrics, estimate techniques, and utility surveys. These are the fifty different methods, and I’ve gone through each one of them. And they all have fundamental problems. They, I’m going through them a bit slowly. Like, here’s one, from the FDA, another benefit risk assessment. Health-adjusted life years are one of the few that uses the same metric for benefit and risk. Number needed to treat is a very popular indication for a single characteristic, but you can’t integrate that across the many factors that needed to do benefit-risk assessment.

And so we’ve gone down the rest of these, methods. If I group these fifty methods by how they accumulate risk, I get a rather useful collection. Most of the methods do not consider all the risk-benefit factors for benefit-risk situation. They will pick on just one factor. And you can’t combine the factors with themselves or with others. It’s simply looking at one factor by itself. So it’s an extremely narrow view of benefit-risk for most of these. The few methods that do look at all the risk-benefit factors, most of them start with what I call the judgment method, where you’re forced to distill all the factors down to the most significant few, only four maybe four to seven methods, four to seven factors.

So either the methods consider only one type of, one factor at a time, or they force you to throw away most of the methods and consider maybe four or seven factors is the second method. The third method is they assign numbers to the factors, they’ll add the factors together, and they’ll divide the benefit sum by the risk sum. And if the division is bigger than one, they’ll say the benefit’s bigger than the risk. And if the division is less than one, they’ll say the risk is bigger than the benefit.

CLICK HERE TO WATCH THIS WEBINAR IN ITS ENTIRETY:
Expert Perspectives: A Method to Assess Benefit-Risk More Objectively for Healthcare Applications

Richard Matt
Latest posts by Richard Matt (see all)