Tag Archive for: safety

Creating a safety-first culture in complex product development

Today’s move-fast-get-it-to-market-yesterday product development culture puts a lot of pressure on companies to innovate quickly. Such circumstances can make defined processes and comprehensive documentation look unsexy and uncompetitive… even when they’re in the best interest of the organization.

In October, Jama Software and kVA by UL co-hosted a kVA Automotive Lunch & Learn at the Hyatt House in Silicon Valley. kVA by UL is a technical and management consulting group focused on functional safety and the ISO 26262 standard. Bill Taylor, Managing Director of kVA, spoke to a group made up primarily of automotive industry engineers, many of whom are working on autonomous vehicles.

Taylor presented on the topic of “Creating a Safety-First Culture in Automotive Development,” but the points he made could easily be applied to any complex product development where public and/or user safety are a primary concern. Here are five key take-aways from Taylor’s presentation.

Don’t Fall for the Smart Folks Fallacy

When we get on an airplane, do we trust that our pilot and co-pilot are experienced, well-trained professionals? Do we assume they really know what they’re doing and that they’ve done it many times before?

Of course, we do. So, why do the airlines — and the military, and every other aviation employer —make their pilots use checklists?

Learn how to mitigate common ISO 26262 mistakes with our whitepaper, “Top 15 ISO 26262 Snafus.”

It’s so they don’t have to think about it. The checklist is there so a distraction doesn’t cause the pilot to miss or forget a small but crucial step in their procedure.

But many who innovate for a living — especially those who face pressure to innovate rapidly — don’t like checklists. Checklists feel cumbersome, tedious, slow, and perhaps even antiquated. They’re constraining. They don’t let you work the way you want. Checklists force you to work to their dictates.

But checklists are great for safety, says Taylor. They force you to take all the prescribed steps.

Taylor warns against a phenomenon he sees often in the automotive and tech industries, which can roughly be described as the “Smart Folks Fallacy.” He describes it with a fictional conversation, which we’ll paraphrase:

“Hey, who’s keeping us safe?”

“Oh, don’t worry, we’ve got some smart folks over there.”

“Yeah, but how do we know we’re safe? Have we written good requirements? Have we analyzed them? Have we done direct testing? Do we have traceability throughout our process?”

“Nah, we haven’t done any of that. But those folks over there are really smart. So, we’ll be fine.”

Unfortunately, even smart people make mistakes. They’re under pressure to deliver. They get wrapped up in their immediate priorities. Even a smart person might miss something that’s tiny but critical.

Safety Relies on Process and Culture

Taylor says it’s hard to standardize culture. A true safety culture requires more than just following a set of rules, regulations, and standards. But a framework can be articulated.

In that framework, Taylor would expect to see a set of processes designed to comply with applicable safety standards. He would want to see process documentation that, among other things, describes:

  • The steps of the safety process
  • Any templates that are to be used
  • The tool set that will be used
  • Outputs that get reviewed, and who reviews them
  • The qualifications necessary to be a reviewer

Failure to follow a sound safety process leads to disaster, says Taylor. He maintains that in every accident report that makes the news, there is someone who says, in effect:

“I realized this could be a problem. I told my boss, and he kind of understood, but not exactly. Anyway, we were crazy busy, etc., etc.… It just got pushed aside.”

You need a system and a culture in place that will turn that potential problem into a documented safety anomaly — that will put it on the radar and make sure it gets addressed, says Taylor.

A Safety-First Culture Needs Police Officers

Just like governments need a police force to enforce laws, companies must invest in safety personnel with the authority to prevent potential hazards from being released into the field.

This doesn’t mean putting safety managers at the top of the pyramid, Taylor says. Instead, he likens it to Toyota’s famous quality system, where every plant worker has the authority to stop the production line if they spot a flaw. It’s a huge expense when it occurs, but it can save much greater expense down the line by avoiding costly recalls and lawsuits. It also makes everyone realize quality assurance is a part of their job.

It’s the same with safety. When a problem is a safety issue, the safety team must have the power to say, “Stop! We have a process to communicate and address this issue. We have the authority to make things stop until the issue is fully resolved.”

Learn how Jama Software worked with TÜV SÜD on our ISO 26262 certification process, and how you can lower the costs and risks of complying with functional safety standards, by watching our webinar.

Embrace the Idea of “Nothing Happening”

Taylor says safety engineers have a weird point of view: They’re inspired by the idea of nothing happening. That’s because, when people buy your product, they expect that nothing bad will happen to them… ever. If something bad does happen to them, and it’s your fault, they’re likely to sue you.

“Someone has to take ownership of this idea,” Taylor said. “There must be a group that says to themselves, ‘While the rest of the organization is innovating and creating amazing things, I’m going to make sure that nothing bad happens, and I can be motivated by that idea.’ It means you’re not out there making the thing be better and more awesome, you’re just trying to make it work in a way that doesn’t kill anyone.”

“And usually,” Taylor continued, “Your CEO says, ‘I thought it was just assumed that it wouldn’t kill anyone.’ And you say, ‘Yes, but someone has to be in charge of that. Someone needs to own that.’”

Accept that Safety is the Antithesis of Agile

While there are varying views on this topic, Taylor pointed out that a safety-first culture is the direct antithesis of the Agile development model.

The Manifesto for Agile Software Development values “individuals and interactions over processes and tools” and “working software over comprehensive documentation,” he said. Safety, on the other hand, cannot be assured without a commitment to process and documentation. “Software observed to be ‘working’ can often be fundamentally unsafe,” Taylor noted.

Taylor went on to say that where Agile tries to reduce friction, safety values friction. Friction is how we maintain control. Without friction, everything slips out of our hands.

“If our process has no friction, it’s out of control,” he said. “Agile is not wrong. Agile is wonderful for making progress” but it’s not sufficient for ensuring safety.

In complex product development, Taylor says, we don’t see a lack of smart people. What’s often lacking is a commitment to process and the tools that enforce that process.

We need to make allowances for Agile and a safety-first culture to coexist. Use Agile to make rapid progress. But allow safety to apply its processes — and apply the brakes, when necessary.

In other words, we need to accept the checks and balances of the safety process. Taylor summed it up this way: “Value friction. Value slowness. Take ownership of ensuring safety.”

To further assist in mitigating risks in the development process, maintaining compliance with automotive functional safety standards, creating a safety-first culture, and staying updated on the changes to ISO 26262, check out our white paper, “The Impact of ISO 26262 on Automotive Development.”

Estimation is a chronically thorny issue for software practitioners. Most people need to prepare estimates for the work they do, but in our industry we don’t do a great job of estimation. In this article I offer several safety tips to keep in mind as you prepare estimates for your project and for your individual work.

Tip #1: A goal is not an estimate.

A management- or marketing-imposed delivery date is not an estimate—it is a goal. A team of a certain size and productivity level can produce only so much high-quality functionality in a given time. Estimation tutorials typically are presented as though the body of work to be done is precisely defined and the purpose of estimation is to determine the corresponding effort, cost, and time needed. Sometimes, though, estimation works another way. If the product absolutely must be delivered by a particular date, then the relevant estimation parameter determines how much functionality of given quality will fit into that time box. Commitments should be based on plausible estimates, not just desired targets. A piece of work should not be considered overdue if there was never any likelihood of completing it by the dictated target date.

Tip #2: The estimate you produce should be unrelated to what you think the requester wants to hear.

If your estimate and the requester’s expectation are too far apart, then of course you must negotiate to reach some agreement. But don’t change your estimate simply because someone doesn’t care for it. Suppose your manager requests a time estimate for some piece of work and you reply, “Two months.” “Two months?” exclaims your manager. “My grandmother could do that in three weeks with one hand tied behind her back!” So you try again: “Okay, how about one month?” What changed in those few seconds? Nothing! The task didn’t shrink, and you didn’t magically become more productive. Your manager just didn’t like your first answer so you offered an alternative that might go over better.

There’s no reason to reduce a thoughtfully crafted estimate simply because someone isn’t happy with it. We don’t expect a rainy day to brighten up just because we feel like going on a picnic (especially in Portland, where I live). Nor does it make sense to alter your own prediction of the future, barring a change in factors that affect how you think that future will turn out.

Tip #3: The correct answer to any request for an estimate is “Let me get back to you on that.”

Avoid the temptation to give someone a quick estimate off the top of your head if you haven’t thought carefully about the problem yet. You haven’t really generated an estimate; you just pulled a guess out of thin air. Unfortunately these casual guesses often sound like commitments to coworkers. Before you provide an estimate, think through what is being requested. Then assess how much time and effort it realistically would take to deliver on that request.

This is a particularly common problem when processing requested requirements changes. A little impact analysis often reveals that the real request is substantially bigger than you might have thought based on the initial information you had available. So before you say, “Sure, no problem,” make sure you know what you’re getting into. Sometimes, providing a more realistic estimate leads the requester to drop the whole thing because it’s just not worth the cost or time required. It’s better to know that before you begin devoting effort on the activity, instead of getting halfway done and throwing out what you’ve already invested.

Tip #4: Avoid giving single-point estimates.

Estimates contain uncertainty; that’s why they aren’t called predictions. When you provide an estimate, also state your confidence in the estimate. Are you ten, fifty, or ninety percent confident that you’ll be finished within the estimated time? We usually don’t even think about that, let alone express it aloud. The person who receives the estimate probably assumes that you’re ninety percent confident in the estimate, even if you’re offering a best-case scenario. This mismatch can lead to unpleasant surprises when the estimate turns out to be vastly overoptimistic.

Alternatively, present an estimate as a range instead of a single value. Identify the minimum possible duration (or whatever measurable factor) for the work, the most likely or expected value, and the maximum expected duration barring some catastrophic event. You can calculate a single, nominal-value estimate by summing the minimum possible value, four times the most likely value, and your maximum estimate for that work, and then dividing the total by six. This results in a weighted average that reflects the probability distribution.

Unfortunately, the people to whom you present a range of estimates might select the lowest value of the range as the one they want to hear. A manager who goes with the minimum estimate has chosen to manage his project at a low confidence level, with only a small likelihood of delivering on project commitments. There’s no defense against unreasonable people who reject thoughtful estimates, but at least get into the habit of acknowledging the intrinsic uncertainty associated with all prognostications.

Tip #5: Incorporate contingency buffers into estimates.

Nothing goes exactly as planned. Therefore, build some contingency time into the estimate for each chunk of work you do. These contingencies will help you accommodate any unplanned tasks that may arise, and they’ll compensate for estimates that turn out to be too low. Contingency buffers also help you cope with risks that materialize into problems. If you don’t leave any slack in your schedule, the first unexpected situation you encounter will demolish your plans. Work to meet the nominal estimates, but commit externally to the estimates and the contingencies.

Contingency buffers are sometimes viewed as simply padding, a fudge factor to protect yourself in case you’re not very good at making estimates. Managers sometimes want to yank out the contingency buffer, assuming that this will somehow shorten the project. A manager who discards thoughtfully planned contingency buffers is making several assumptions:

• All estimates are perfect.

• No risks will materialize into problems.

• There will be no growth in the requirements.

• Nothing unexpected will happen.

Anyone who has ever worked on a software project knows that these are not very realistic assumptions. To help a manager avoid these terrible assumptions, you should point out some unexpected experiences from previous projects. Ask the manager if there is any reason to believe that the new project will be different and that none of those unpleasant experiences will be repeated. If no reason can be provided, then the contingency buffers should stay.

These five safety tips might not help you generate estimates that all of your managers, customers, and coworkers will dance to, but they will help you and your team at least hear the same music.

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.