I am writing this post about reducing the risk of negative outcomes in the product development process in the hopes of saving at least some of you from the fate that has befallen many companies before they reach out to us. Unfortunately, many of our new clients come to us AFTER they‘ve had a negative product outcome at their current company, or even more common, at their new one.   

According to an Engineering.com survey (full disclosure: we sponsored it) 83% of those companies surveyed experienced at least one negative product outcome including: significant delays, cost overruns, product defects, compliance gaps, recalls, omitted requirements, and lengthy rework. In many cases these negative outcomes were quite significant and led to changes in management and staff.   

Root Causes of Risk in the Product Development Process 

All of these companies conducted root cause analyses to determine what led to these negative product outcomes. The root causes identified are quite similar across the organizations: 

• Limited customer and cross-functional involvement in the review and approval of requirements 
• Static requirement documents rarely viewed by key stakeholders, maintained in Microsoft Word/Excel or a standalone tool, and used only be a few as a repository 
• Missing decomposed requirements  
• No ability to track the life of a requirement through development, test and release 
• Release cycle misalignment across engineering disciplines 
• Misinterpretation of requirements across engineering disciplines 
• No process exception tracking to determine requirements that have been omitted or modified 
• No identification of process risk patterns – delays in development, multiple test failures, rework cycles, etc. 

Product Development Process is Not Under Control 

With all of these informational and process gaps, the end-to-end product development process is “not under control”, to extend the concept of Six Sigma statistical process control to a complex process. A process that is not under control cannot be relied upon to deliver predictable results. The outcomes can vary widely and with no ability to truly monitor risk probabilities as the process moves along, the negative outcome occurs as a surprise at the end of the process where the cost is highest.  

Many companies operate under the illusion of risk mitigation. The process is “not under control”, so management relies on managing people and takes each team lead’s assessment of progress and risk as comfort that the risk of a negative outcome is low. Unfortunately, the team leads are only assessing their teams and what they see. This misses the most likely points of failure that connect teams and extend over time. Without Living Requirements™ it’s hard for anyone to see across all teams (and time) to identify the exceptions, omissions, defects, rework, delays and dependencies.  

Even organizations more mature in their end-to-end product development process management are still often surprised by negative outcomes.  A very thoughtful root cause analysis of an actual situation of this type has been done by Michael Panis and presented at the 2020 IEEE 28th International Requirements Engineering Conference (RE). His key findings include:  

• “Poor traceability and missing subsystem requirements when a system requirement is decomposed through an error budget or other complex logic.” 
• “Missing requirements due to a decision to only trace to subsystems directly responsible for given system requirements.” 

ROI from Risk Reduction in the Product Development Process 

Investments to improve the product development process are typically justified through gains in engineering productivity. What is often missed is the even greater benefit to the organization of bringing the product development process under control and reducing the probabilities of negative outcomes. A basic approach is to quantify the magnitude of the various negative outcomes — and the likelihood of their occurrence — to determine the current risks in financial terms that could be faced by the organization. The next step is to estimate the reduction in occurrence probability given the improvements in the process and then calculate the overall reduction in the magnitude of the expected negative outcomes.  A more advanced model is described in the paper by Machac, Steiner and Tupa.   

A Common Mistake to Avoid 

Let’s wrap up with a common mistake many make when measuring risk in the product development process. There is an obvious difference between the likelihood of a negative outcome and the financial impact if the negative outcome occurs as the graphic below shows. 

However, many confuse a constant likelihood of a negative occurrence (scenario A – unmanaged risk in the graphic) with a constant level of financial impact when, in fact, it is growing dramatically throughout the product development process as investment and costs to remediate omissions, defects, and reputation grow over time. The only way to maintain a constant and low level of risk is to actively manage down the likelihood of a negative outcome continuously through the development process as shown as scenario B in the graphic. For a deeper dive into this concept please check out a fine piece by Preston Smith in Research- Technology Management, Managing Risk as Product Development Schedules Shrink, and the source of the graphic above. 

Watch this webinar to learn more about moving away from documents-based design control and risk management.

 WATCH THE WEBINAR