Product Development & Management

Manage by Exception: Data-Driven Practices to Improve Product, Systems, and Software Quality

By |
Manage by Exception

In this blog, we recap our eB00k, “Manage by Exception: Data-Driven Practices to Improve Product, Systems, and Software Quality” – Download the complete paper HERE.

Manage by Exception: Data-Driven Practices to Improve Product, Systems, and Software Quality

Requirement errors in product development cost time and money and create potential liabilities. The expense of these errors can make up between 70% and 85% of all rework costs. When leaders don’t have data related to the execution process, teams aren’t tracing requirements back to the “‘why’,” and when there’s a lack of insight into aspects like verification coverage, you’re much more likely to encounter programs late in the development cycle, resulting in expensive problems.

This creates the all-too-familiar scenario seen in the news of product, systems, or software defects and the resulting fallout. Organizations can avoid many of these challenges by accessing the right data at the right moment — and ideally early — in the development process. As most executives and managers know, you can’t manage what you can’t measure. Using data to measure allows your teams to spot recurring patterns and abnormalities early, before they grow into larger challenges later in the development cycle.

Requirement errors in complex product, systems, and software development can consume between 70% and 85% of all project rework costs.

Why “Data-Based Management” is Critical, and How it Uncovers Gaps

Management by exception is a method that empowers your team with data focused on early warning indicators. It’s these warnings that help support faster and more informed decisions.

As a result, leaders can focus on exceptions rather than needlessly micromanaging and intervening with teams when the data shows that development is going as expected.

In other words, when using data, the goal isn’t to micromanage, but to do the opposite: leverage the data to do less micromanagement.

The result is fewer manual requests, fewer status updates, fewer test procedure specification reports, and fewer unnecessary meetings.

Data-driven practices help you automatically evaluate exceptions without needlessly relying on a person to manually hunt them down, evaluate them, and communicate about them. Instead, abnormalities and oversights are brought forward to reduce managerial workloads by minimizing unnecessary intervention and allowing more time to be spent in areas that have the greatest impact.

RELATED: Buyer’s Guide: Selecting a Requirements Management and Traceability Solution

Examples of Exceptions in Daily Product, System, and Software Development and Requirements Management

As you adopt a data-driven approach, there are several considerations, but the first is identifying the expected or acceptable process for your research and development function.

Many organizations don’t have a defined practice; instead, operations are based on how things have always been done. Defining processes gives you greater focus.

Once you have an expected process, you can leverage the data to manage by exception but also take things a step further by managing requirements quality, traceability, and completeness. These capabilities will help predict and prevent poor outcomes in product, system, and software delivery.

A tool such as Jama Connect® can help you successfully manage exceptions, such as in these four examples.

1. Version and Change Management – The Jama Connect dashboard shows requirements missing verification. For example, it might flag two requirements missing verifications, and if you click for more details, you can view a filtered list of those requirements. And you can ask Jama Connect to show those missing a downstream verification. The filters are a powerful way to understand and create audits for capturing those exceptions in your process.

2. Derived Requirements Missing Rationale – Using a filter, Jama Connect allows you to see if a particular requirement has a missing rationale. For example, a hardware or software engineer may create a new requirement. But when that’s done,
it’s crucial to have a solid rationale for why, especially if the requirement is not directly related to a stakeholder’s need or contractual requirement. You don’t want to introduce unnecessary capabilities that aren’t going to align with the actual user needs or have a real rationale behind them.

3. Remediate Rejected Requirements – Jama Connect has a capability called Review Center. It allows you to send requirements into a review with colleagues, which can increase the quality of the requirements and create a shared understanding. Leaders can quickly spot the rejected requirements and discuss how to move forward. With many organizations working remotely, this capability helps increase asynchronous collaboration so that working sessions and meetings can more efficiently focus on exceptions.

4. Find Poorly Written Requirements – The International Council on Systems Engineering (INCOSE) created a handbook of recommended rules to author well written requirements. For example, requirements using vague terms that are not testable could be flagged for improvement.

With Jama Connect Advisor™, powered by natural language processing, INCOSE’s best practices, and the Easy Approach to Requirements Syntax (EARS), teams can now check the quality and accuracy of their requirements.

RELATED: Requirements Traceability Diagnostic

Critical Metrics to Consider

Having data, a way to view it in context, and metrics to track it empowers your leaders to make the right decision at the right time and predict how well the project or product development will trend. Metrics are an essential part of that equation, and here are two to consider tracking.

1. Requirement Quality – Most product, system, or software failures are due to undocumented, poorly written, or misunderstood requirements. And the later in the product lifecycle the problem is discovered, the higher the cost. Measure your requirement quality, and if you need support, your Jama Software team of in-house experts can help with audit assessments, training, and other resources to help improve the quality of your requirements.

2. Traceability Score™ – Traceability is a core tenet of building complex products, but it hasn’t been measured in a standard way in the past. But if you can measure it, you can improve it.

For example, Jama Software has aggregated and anonymized over 40,000 projects and over 6,000 traceability models using Jama Connect. And we’ve defined an actual approach to measure a Traceability Score™.

Our Traceability Benchmark study shows this traceability score produces a clear correlation between quality and time to market.

It starts with setting up the expected behavior of your engineering team – the traceability model. We take the number of established relationships among the different model elements in the traceability model and divide that by the number of expected relationships defined by the project’s relationship and traceability model. This gives us the traceability score.

For example, a requirement should have three different elements:

Imagine an example where you have two of those established, but one is missing. The Traceability Score is 66%, and with that metric, you can take the appropriate action. Our above-mentioned benchmarking research showed that higher Traceability Scores™ equaled improved product quality and faster time to market.

Integration of your digital engineering tool suite is critically important. In product development, many tools such as Excel, development applications, modeling applications, testing applications, and others are used. These tools capture critical data about your product and system development lifecycle.

But if they aren’t integrated, you can’t measure critical information like your Traceability Score. As a result, managing by exception isn’t possible due to a lack of data, which risks product delays, extra costs, and even compliance and audit failures. Ensure that critical tools are integrated to support real-time data visibility.

TO LEARN MORE, DOWNLOAD THE COMPLETE WHITEPAPER HERE:
“Manage by Exception: Data-Driven Practices to Improve Product, Systems, and Software Quality”

Jama Software
Latest posts by Jama Software (see all)