Traceability

Unlocking Development Success: Jama Connect® vs. Word & Excel for Requirements and Traceability

By |

This image portrays a person unlocking their product development with Traceability vs Excel

Proactive and Live Traceability™ in Jama Connect® vs. Retroactive and Lagging Traceability in Excel

Incremental increases in product development complexity can lead to an exponential increase in the effort required from Engineering and Research and Development teams to keep up in a document-based (Word/Excel) environment.

PROBLEM

Our medical device & life sciences team has thousands of conversations per year with people interested in product and systems development improvements.

When we look at customer data over the two years prior to adopting Jama Connect, 83% of these organizations had their traceability maintained in an Excel-based matrix.

In this environment, all of the traceability components are maintained in a separate system (in other documents or tools). What we found was that this led to traceability being disconnected from the actual design. And in these cases, this delta was maintained and updated manually.

Updating the Excel-based traceability matrix to reflect changes or new artifacts is always a manual process.

Because this process is not automated, it takes a significant effort and is also highly error-prone. On a small scale, it can be manageable. But once a change is made, managing it effectively in this way becomes a significantly greater problem. We found that these events can exponentially increase both the level of effort needed to maintain traceability and the risk of a negative outcome or occurrence. We’ll provide some examples of this shortly.

As organizations make incremental improvements and changes, or grow and scale the company further, the manual, Excel-based process can become a major bottleneck. Because the level of effort associated with changing this workflow can seem too big of a task to complete, process improvements are de-prioritized. We see an exponential scale difference between an increase in complexity and the difficulty in managing traceability in Excel manually – tightening this bottleneck. Even a slight increase in complexity can lead to a high-severity issue/business impact/time waste.

As an example, let’s take a simpler medical device (a single-use catheter). Here are a few examples of how you could have the traceability schema established:

  • 1-2 levels of requirements traceability (e.g., User Need > Product Requirement > Verification, or User Need > Validation)
  • Few items per level at each level in the hierarchy (e.g., 5 User Needs, 10 Product Requirements, and 1-2 Verifications/Validations)

As described in the example below, this means for simple products like a single-use catheter, there are roughly 225-440 possible trace relationships:

Let’s now imagine that we want to make a seemingly simple change and additional functionality to the device. We want to connect the catheter to a mobile application, so it can monitor its usage and analyze it for diagnostics purposes.

As this device is now part of a “system” with two major sub-system components, we break down the “product requirements” into two separate levels: “system” and “sub-system” requirements.

The complexity associated with managing requirements and maintaining traceability increases exponentially. For example, we’ve outlined a common example of a Class II system, where you can see that a 4x – 10x increase in the number of User Needs translates to a 15x – 24x increase in the total number of trace relationships that need to be managed.

An increase of 4x-10x in the number of User Needs translates to a 15-24x increase in the total number of trace relationships that need to be managed.


Here are a few common real-world scenarios where we see this complexity change occurring; A couple of examples of other complexity increases for a medical device organization:

  • A medical device with a newly added component and functionality (software, electrical, mechanical, hardware, etc.)
  • A research use application (RUO) that is reclassified as an IVD – becoming a diagnostic device
  • A medical device startup going from an R&D phase into having a product out in the market
  • A medical device being implemented into a larger system
  • Expanding into different markets and adhering to new regulations (US FDA, EU MDR, etc.)
  • Introducing product families, product lines, and variations to more effectively reuse existing components

The more this process/tool improvement is delayed, the higher the risk to the business.

We see this happening to medical device companies both big and small. Here are a few of our customers describing the problem.

RELATED: The Strategic Transition: From Word and Excel to Modern Requirements Management

Microsure

“With Word and Excel, if something is changed and a link is broken, that document is gone and it’s literally floating around somewhere in the cloud without linkage to anything. This makes it very scary, especially from a quality or regulatory perspective. Our Word and Excel process evolved with the organization and therefore it was put together layer by layer, making it really hard to have the full depth of knowledge about how the quality system works.” – Rene Wenmekers, Director of Quality & Regulatory, Microsure

“We work in a highly regulated environment, and Microsure’s product has hundreds of requirements on system, subsystem unit, and component levels. And from a regulatory documentation standpoint, information is scattered.” – Robin Brounds, Software Team Lead, Microsure

“When we make changes in medical device development, they need to be reported to the notified body. And when that change hits the level of ‘significant change,’ the whole documentation set needs to be provided to the notified body to be reassessed on safety and efficacy. Every time a requirement changed, it needed to be updated across the whole documentation path. This was not sustainable using Word and Excel, and it was risky.” – Rene Wenmekers, Director of Quality & Regulatory, Microsure

Convergent Dental

While using Word and Excel, Convergent Dental found themselves tracking across multiple documents, all with their own trace matrix tables relating to different requirements. The fallout from this process is that even a single word or letter change in a low-level subsystem requirement led to updating corresponding requirements documents in their trace matrix tables. So, a single letter turns into not one change but potentially six changes across five different documents.

“We have a small team with a large amount of features and updates to perform on an ongoing basis. We all work really hard here, and there’s no option to be dead weight. Getting rid of that wasted time in Word and Excel and getting our test engineers back to work is the ultimate goal.” – Craig Woodmansee, Electrical Systems Engineer, Convergent Dental

NEGATIVE BUSINESS IMPACTS – of not changing

Wasted Time and Inefficient Processes

  1. In a complex setting (working with a complex product, highly regulated environment, high-risk product, cross-functional teams working together, lots of different product variations, target customers/markets, etc.), this can be a person or a team’s full-time effort to keep up to date.
  2. Time is being spent on people trying to find the right and most up-to-date documents
  3. Sitting through review and alignment meetings with all stakeholders

Increased Risk of Negative Outcomes
This process relies on people constantly monitoring and updating each change. If a change goes unnoticed or people forget to update traceability, this gap is difficult to notice. If traceability gaps are noticed later during the product development lifecycle, there is a significant increase in the risk of one of the following negative events happening:

  1. Releasing a faulty & untested product, quality compromise, product callbacks
  2. Forcing organizations into late-stage changes that are costly to implement
  3. Regulatory issues and audit findings (non-conformities, FDA warning letters, etc.)
  4. Product not meeting the original requirements and customer/stakeholder needs

The expected outcome – backfilling documentation and traceability at the very end of the project.

Real outcome – Many issues/gaps went under the radar, leading to project delays, missed deadlines, or regulatory/quality issues:

Decreased Organization Maturity, Disconnected and Siloed Teams

  1. Enforce the defined process – In a document-based environment, it’s close to impossible to monitor and enforce the defined process
  2. Impact on employee tenure – Engineering and R&D are forced into manual documentation instead of actual design & development
  3. Impact on talent acquisition – High-quality talent is more attracted to companies with proper tooling and processes in place
  4. Communication and Transparency – Audit trails and change logs are often lost, hard to keep people accountable for changes

RELATED: Loram Rides the Fast Track to Software Safety with Jama Connect®

SOLUTION

The solution to this problem is having integrated risk management with Live Traceability™ in Jama Connect®. Jama Connect will be the overarching system across all product development initiatives, bringing together all disciplines, making it significantly easier to visualize complex traceability hierarchies, replacing the manual effort needed to keep the documentation up to date, etc.

Jama Connect® brings comprehensive and detailed insights into your complex product, systems, and software development processes – automating the measurement of requirements traceability and coverage across disciplines and your organization’s toolchain.

This level of visibility helps eliminate rework due to out-of-date information; and the biggest fear for engineering leadership – that the greatest risks to a project are unseen until it is too late.

Adam Gor
Latest posts by Adam Gor (see all)