Tag Archive for: Product Development & Management

requirements-driven testing

Jama Connect® vs. IBM®DOORS®: Requirements-Driven Testing: A User Experience Roundtable Chat

Increasing industry challenges and complexities are pushing innovative organizations to consider modernizing the tool(s) they use for requirements management (RM). In this blog series, Jama Connect® vs. IBM® DOORS®: A User Experience Roundtable Chat, we’ll present several information-packed video blogs covering the challenges that teams face in their project management process.

In Episode 9 of our Roundtable Chat series, Mario MaldariDirector of Solutions Architecture at Jama Software® – and Susan ManupelliSenior Solutions Architect at Jama Software® – discuss requirements validation, verification, and testing in addition to demonstrating test management in Jama Connect.

To watch other episodes in this series, click HERE.

Watch the full video and find the video transcript below to learn more!


VIDEO TRANSCRIPT:

Mario Maldari: Hello, welcome to the ninth edition of our vlog series. Today, we’re going to be talking about something that’s very important in requirements management, something that I’m particularly passionate about, and that’s requirements validation, verification, and testing. And I’m joined by my friend and colleague once again, Susan Manupelli. Susan and I have worked together for a long time, 15 years plus testing various requirements management tools using various techniques, and various software. I believe the most recent software you were using was IBM’s enterprise test management tool, something we used to call RQM. Looking back on all those years and all those tools you feel as though have been your biggest challenge.

Susan Manupelli: So talking about the ELM suite where we were talking about rational quality manager and also we were using that to test DNG. Really the issue, the biggest challenge is that they were two separate tools. So even though they were part of the same tool set, the UIs were completely different. They were very inconsistent in how you would use them. The review and approval aspect of RQM wasn’t that great. And again, it was completely different from the review and approval that you would get when you were working with DNG. And also because they were from two separate tools, in order to really get the traceability, that would be a challenge. You’d have to do reports that were outside of the individual tool tools. And then one of the biggest things too was the comparison. Things changed in RQM. It was not easy to find out what changed, even if you compared one test case to another.

Mario Maldari: Yeah, I recall some of those challenges. I think for me, the biggest challenge I had was the UI inconsistencies like you mentioned. Obviously, I was in one tool, I’d go to another. It’s completely different experience, completely different nomenclature. And then having to integrate between the tools and just frankly having to go to a separate tool to do the testing was problematic and challenging sometimes. So I think you hit an important topic in terms of having everything in one tool. And I’d like to show you how Jama does that. Okay. So in Jama, the fact that we have testing integrated into the tool allows you to do some pretty neat things. So as you can see here on my desktop, we have this dashboard, and I can define a relationship rule diagram in Jama where I can define that I want specific requirements to have validation points and test cases associated with them.

And so what that gives me is I can create some dashboard views for requirements, lacking test coverage, or I can even look at test case summaries. Right on the dashboard, I can look at test case progress, the priority of my tests. Jama even allows you when you’re testing to log defects. So I can track my defects here. And so for you and I, we always have to provide test case reports and summaries up through management, up through the development team. And so this allows you to have it all in one spot, which is really nice to have. So the testing itself in Jama, you basically enter it on the test plan tab and very similar to the way you and I worked, we have a concept of a test plan where you can define your test intent, the things you’re going to be testing, your approach, your schedule on your team, your entry criteria, your exit criteria.

And from there, as you pointed out, you can send this for a review and you can get an official sign-off from your development team or whomever you need to sign off on your test plan. And then once that’s in place, you can go to your test cases and you can start to group your tests according to functionality or whatever makes sense for your grouping and your organization of your suites of tests. And once they’re grouped, you can come to the test runs and this is where you actually will be doing your execution of your test. So I can click on one of these here and can start an execution and I can start to go through each step and pass or fail as I go through. And the nice thing about Jama, as I mentioned, is that you can actually go ahead and log a defect in real time and I can go ahead and log this defect.

And now when I’m saving this defect, it’s associated with this test execution run, which is associated with my test case, which is associated with multiple levels of requirements upstream. So now if I look at a traceability view, I will see my high level requirements traced all the way down to the defects. When I have logged a defect, I can actually go in and I can take a look at this test run and I can see the defects. And if I have something like an integration to another product like Jira for example, maybe my development team and is working in Jira and they love Jira, it automatically populates the defect in the defect tool like Jira. So a developer can come in here, they can make some changes, they can put in some comments, they can change the priority, the status, and all of that gets reflected back in Jama.


RELATED: Traceability Score™ – An Empirical Way to Reduce the Risk of Late Requirements


Mario Maldari: So really nice integration if you’re using something like Jira. From my perspective too, what would’ve been nice in my past test background is to have this concept of suspect trigger. And so if I look at the relationships for this particular requirement and I see that downstream there’s a validation of a test case, which is validated by length type, I can see that it’s flagged as suspect. So that means that something upstream has changed and my downstream test case is now suspect. And what does that mean? Maybe I need to change it, maybe I don’t. How do I know? I can come to the versions and I can say, “Well, the last time I tested this requirement was in our release candidate one, and what’s different now?” So I can compare our version three to version seven, run our compare tool, and I can see exactly what changed.

So as a tester, this is great to me, it’s not enough to know that something’s changed. I can actually see exactly what changed and maybe it’s just a spelling update and I don’t need to really change it. Or maybe it’s something more substantial like you see here. And at this point I can come in and I can make my change to my test and I can go ahead and I can clear the suspect flag.

So really nice level of granular control. What’s also good with the Jama’s we have these out of the box, and you’ll like this, Sue, out-of-the-box canned reports that have summaries of your tests, how many blocked, how many failed, how many passed executions. So these are canned reports that come with Jama. If you needed any customized reporting for your specific needs of the organization, we have that available as well. So really nice back to your point about having everything in one tool, this is it, and this is the benefit. Now, I know you’ve been at Jama for just about six months now. I’d love to hear your impression of the test management built-in, what your thoughts are there?


RELATED: Telesat Evolves Engineering Requirements Management & Product Development


Susan Manupelli: Oh, sure. Yeah, I do. I definitely love how everything’s in one tool and the ease with which you can just trace, actually verify the testing of your requirements. You can just go from requirements straight down to multiple levels of decomposition to your test cases. So you can see, answer the question, did your requirement are your requirements passing, which is great. And also the ability to display related queries right on the dashboard. I think that’s a huge plus the consistency of the UI between what you do for requirements, creating a test case isn’t any different than creating any other requirements.
So it’s a very familiar UI for both operations, which I think is important. The review and approval piece is really a nice strong point for Jama, and to be able to apply that to reviews for test cases is really great. And I just think it’s a really streamlined UI. It really has everything you need and nothing that you don’t. So I just think it’s a great tool. And then there’s one other aspect that I really like is the impact analysis. You mentioned being able to trace when something’s changed after the fact. It’s also to be able to say, “Hey, we’re looking at making a change here.” There’s one button in Jama, you click that impact analysis and it tells you all of your test cases that you might need to revisit if you make that change.

Mario Maldari: I call that the proactive method.

Susan Manupelli: Yes.

Mario Maldari: Yeah, the impact analysis is extremely important. And if you were a developer in an organization and you changed a requirement or you were about to change a requirement and you knew you had 30 tests that are associated with that, you could run the impact analysis. See all of those, and you could proactively warn your test team, “Hey guys, I’m about to make this change. Here it is. I’ll explain it to you. We can have a separate review and approval.”

So it really contains all of that and controls all of that for you. I’ve often said to people, it’s one thing to have your requirements in a tool, and that’s the first step. Define your requirements, have your traceability. But if you’re not doing your testing and validating those requirements, then how do you know that you built the right thing, right? So extremely important aspect testing to requirements in the supply. So any requirements gathering process so I’m glad we could talk about it today. Sue, glad I could have you to talk to about it. And I’d like to thank everyone for their time and thanks for participating in the vlog series and we’ll see you on the next one.


Is your data working for you? A consistent and scalable data model is instrumental for achieving Live Traceability™ and making data readily available across the development lifecycle.

Download our Jama Software® Data Model Diagnostic to learn more!


Thank you for watching our Episode 9, Jama Connect vs. IBM DOORS: Requirements Driven Testing. To watch other episodes in this series, click HERE.

To learn more about available features in Jama Connect, visit: Empower Your Team and Improve Your Requirements Management Process



trace score

Trace Score™ – An Empirical Way to Reduce the Risk of Late Requirements

Executive Summary

One of the main causes of rework, delays, and cost overruns in product development is the creation of new requirements late in the process. This is a well-known risk in product development, but what management practices can empirically be shown to reduce this known risk?

Using our proprietary database of metadata from over 50,000 complex product development projects, we were able to determine that the Trace Score is an empirical method to reduce late requirements. In fact, teams that maintain a high Trace Score reduce the burden late requirements have on their project by 67% compared to teams with low trace scores.

  • With this knowledge, our recommendation is that practitioners measure and monitor the Trace Score of their projects to resolve issues early and ensure that the risk of late requirements is kept to a minimum.

Dataset Background

Jama Software has the world’s largest, live dataset of engineering process performance with over 50,000 engineering projects updated and growing continuously. Leveraging this dataset, it is now possible to determine empirically which management practices improve the performance of the product development process. To learn more about our benchmarking, please review our Traceability Benchmarking Report.

The Empirical Questions

In this analysis we will explore three key questions:

  1. What are late requirements?
  2. How do late requirements negatively impact projects?
  3. Does maintaining a high Trace Score reduce the risk of late requirements?

What are late requirements?

For the purpose of this analysis, we define “late requirements” as those requirements created after the completion of a project’s requirement decomposition phase which we estimate as spanning the middle 50% of all requirement creation activity (creation and refinement). To illustrate what late requirements look like, we show two actual projects below with requirement activity plotted over time.

Requirement Creation Over Time

 

In the Timely Project, requirement creation occurs in a defined requirement decomposition phase to form a necessary and sufficient set of requirements, with very few requirements being added after the fact (e.g. in fig (a), only 1.3% of requirements created late). In the Late Project’, requirement creation bleeds into future phases of the project, leading to a significant amount of late requirements (e.g. in fig (b), 9.2% of requirements are created late).


RELATED: Requirements Traceability Benchmark


How do late requirements negatively impact projects?

We can measure the outsized burden late requirements have on project teams, which we have illustrated for our two projects below. We define late requirement burden as the total number of requirement activities (creation and refinement) attributed to late requirements as a percentage of all requirement activity.

Impact of Late Requirements on Project Team Activity Burden

In the Timely Project, minimal late requirements enable better forecasting of project completion and limit the rework and cost brought on by late requirements (e.g. in fig (c), late requirements only create an additional 8% burden).

In the Late Project, the high volume of late requirements makes it much harder to forecast project completion as the scope of the project is constantly changing, and project teams need to accommodate the late requirements (e.g. in fig (d), late requirements contribute an additional 31% burden).

Unsurprisingly, this additional burden of late requirements has an impact during testing for requirement validation. In our actual project examples, the Late Project has a test failure rate over 3x that of the Timely Project.

percentage chart


RELATED: Unlocking The Power of Live Traceability with Jama Connect


Does maintaining a high Trace Score reduce the risk of late requirements?

A core theorem of Systems Engineering is that maintaining high requirement traceability from the start of a project reduces the risk of late requirements and negative product outcomes. With our project dataset we can now test this theorem empirically. We define traceability as a measure of a project’s ‘expected’ traceability that has actually been established and calculate the Trace Score as follows:(1)

established over expected

For our example projects, the Timely Project achieved a Trace Score over 6X that of the Late Project; suggesting that maintaining a high Trace Score throughout the project reduces the risk of late requirements.

traceability chart

To further determine if Trace Score correlates to late requirements, we divided our dataset of projects into quartiles based on their Trace Scores (Quartile 1 = bottom 25% trace score, Quartile 4 = top 25% trace score) and then compared the distribution of ‘Late Requirements Burden’ across these quartile groups. What we found is that projects within the bottom traceability quartile had a median Late Requirements Burden 3x greater than those in the top traceability quartile. In other words, the evidence supports that projects managed with higher traceability generally experience less risk from late requirements.

Recommendation

Our analysis has shown that late requirements negatively impact projects and that managing projects through a Trace Score is the only empirical way to reduce the risk of late requirements. Below you can see how one can measure the Trace Score over time as a project progresses to ensure system engineering best practices are being followed. A low or falling Trace Score can quickly identify areas to address to reduce the risk of late requirements.

Here you can see how managing the Trace Score directly as the project is underway would have identified the risk early in the Late Project.

Benchmark Chart

To learn more about achieving Live Traceability™ on your projects, please reach out for a consultation.

Interested in learning more? Download the entire Research Notes: Trace Score datasheet HERE.

 



Jama Connect® Features in Five: Document View

Jama Connect® Features in Five: Document View

Learn how you can supercharge your systems development process! In this blog series, we’re pulling back the curtains to give you a look at a few of Jama Connect®’s powerful features… in under five minutes.

In this Features in Five video, Katie Huckett, Senior Product Manager at Jama Software®, walks viewers through Document View, a new feature offered in Jama Connect.

In this session, viewers will learn how Document View, now available alongside list and single-item views, allows users to:

  • Author, read, and edit items in line in a single view while maintaining an item-based structure within project hierarchies.
  • Improve consistency and accuracy of requirements quality by incorporating built-in support for Jama Connect Advisor™, an add-on to Jama Connect.

Jama Connects complete requirements authoring solutions supports different use cases and different preferred user work styles such as those previously performed in siloed tools like Microsoft Word or Excel.

With Document View, you can leverage all the functionality and toolbar actions of reading view, such as filtering and configuring items, reuse, batch transition, send for review, edit and more. Double-click on an item to open quick edit mode with the option to expand to full edit mode. Insert new items without losing your place in the document, add comments and lock or unlock items.

Follow along with this short video below to learn more – and find the full video transcript below!


VIDEO TRANSCRIPT:

Katie Huckett: Hi, my name is Katie Huckett and I’m a senior product manager here at Jama Software. In this video, I’m going to walk you through Jama Connects new feature Document View. Jama Connect now provides Document View, alongside list and single item view. Document view allows users to author, read and edit items in line in a single view while maintaining an item-based structure within project hierarchies. Document view improves consistency and accuracy of requirements quality by incorporating built-in support for Jama Connect Advisor™ and add-on to Jama Connect. Jama Connects complete requirements authoring solutions supports different use cases and different preferred user work styles such as those previously performed in siloed tools like Microsoft Word or Excel.

With Document View, you can leverage all the functionality and toolbar actions of reading view, such as filtering and configuring items, reuse, batch transition, send for review, edit and more. Double-click on an item to open quick edit mode with the option to expand to full edit mode. Insert new items without losing your place in the document, add comments and lock or unlock items.

Let’s see what this looks like in Jama Connect. Here in Jama Connect, I wanted to start on the current reading view so that you can see as I toggle over to our new Document View the transition to the new, clean, modern design. We’ve removed the horizontal lines between the items for a more seamless document experience. The item ID and current version are visible under the item name and comments and locking functionality have moved to the right of the item name so they don’t get lost within the content itself. Use the edit feature to quickly edit items without changing views or manually tracking your place in the document. I’ve opened what we call quick edit mode, which is a condensed form of fields only visible on the current view, as well as any additional required fields that you may have missed that need to be completed in order to save the item.

If you need to see the additional fields available for this item, expand to full edit mode and then you’ll be able to access any additional fields that you need. Quickly return to quick edit mode to complete any edits that you need before saving and completing your work. As I mentioned previously, Document View provide support for Jama Connect Advisor™. As you highlight text in a rich text field that you have enabled advisor for, you’ll notice an analyze button beneath the field. As you analyze the results, you’ll then see any recommendations that have been found. Click the view details button to see the information in more detail.


RELATED: Jama Connect® Features in Five: Jama Connect Advisor™


Huckett: Create new items and Document View with our new inline insert. I’m going to insert a new item between item one and two here, so I have a new requirement that needs to go in here. So you’ll see as you hover between the items, you have a plus button for inline insert form, and I’m going to go ahead and insert a new design description. You’ll notice that our inline insert form is very similar to the quick ad functionality that’s available in the ad dropdown in the content header. Only the name and description fields are visible, name being the only one that’s required. We are bypassing any additional required fields at this point so that you can quickly add as many items as you need to and then go back and edit in more detail and fill out the remaining required fields.

So you’ll notice I’ll add in a name and description into this item. You’ll note the Jama Connected Advisor analysis is also available in the inline insert functionality. We’re going to save this item. You’ll receive a toast message that lets you know your item’s been created, and you’ll see that new item appear in between items one and the previous item two that I had before. So as I mentioned, there is an additional required field on this item that I did not complete before. So I’ll go back in, edit this item, find that additional required field and assign someone to it so that we can then fully save and complete this item for the time being.

In order to view comments, you’ll click on the comments icon next to the item name. After clicking on the icon, you’ll see the comments stream up here in a modal above Document View where you can interact with, comment and reply to any comments on the item. Next, I’ll take you over to the admin section for your Jama Connect administrators to customize and configure Document View and Jama Connect Advisor™ to your organization’s needs. For each item type, it can be configured for default Document View settings. You’ll find a new projects Document View option in the view dropdown where you can then place your default visible fields. Jama Connect Advisor™ can be turned on for any rich text field on any item type your organization chooses and left off for any item types that don’t need the analysis.


RELATED: Jama Connect®: Quick ROI Calculator


Huckett: When you open a rich text field on an item type, you’ll notice a brand new checkbox for Jama Connect Advisor™. Enable advisor for that particular item type field and save your configuration either before or after the individual item field configuration for Jama Connect Advisor™. Don’t forget to go into the dedicated admin section to enable the INCOSE rules in whole or selectively based on your needs and the EARS patterns.

For more information about Document View, please contact your customer success manager or Jama consultant. And if you would just like to learn more about how Jama Connect can optimize your product development processes, please visit our website at jamasoftware.com. Thank you.


To view more Jama Connect Features in Five topics visit: Jama Connect Features in Five Video Series


x


Jama Connect® vs. IBM® DOORS®: Reuse and Variant Management: A User Experience Roundtable Chat

Jama Connect® vs. IBM®DOORS®: Reuse and Variant Management: A User Experience Roundtable Chat

Increasing industry challenges and complexities are pushing innovative organizations to consider modernizing the tool(s) they use for requirements management (RM). In this blog series, Jama Connect® vs. IBM® DOORS®: A User Experience Roundtable Chat, we’ll present several information-packed video blogs covering the challenges that teams face in their project management process.

In Episode 8 of our Roundtable Chat series, Mario MaldariDirector of Solutions Architecture at Jama Software® – and Gary HayesSenior Solutions Architect at Jama Software® – discuss the importance of reuse and variant management for product teams.

To watch other episodes in this series, click HERE.

Watch the full video and find the video transcript below to learn more!


VIDEO TRANSCRIPT:

Mario Maldari: Hello everyone. Welcome to episode eight in our vlog series. Hope you’ve enjoying the series so far. Today, we’ll be discussing the important topic of requirements reuse, and I’m joined by my friend and colleague Gary Hayes today. Gary, would you like to introduce yourself?

Gary Hayes: Sure thing, Mario. Thank you. My name’s Gary Hayes. I am part of Mario’s team. I’ve been working with systems and software engineering teams for the last 25 plus years, just recently have joined Jama. Prior to that, I spent 18 years working with Rational Software and IBM, supporting their systems and software engineering tools suite. Got a lot of experience in the early days with Requisite Pro as part of Rational Software before being acquired by IBM and then, morphing into the Jazz-based tools and Requirements Composer, acquiring Telelogic and DOORS, and then, DOORS Next Generation. So, been around tools for a long time.

Mario Maldari: 
Thanks Gary. You’re like me. We’ve been in Requirement Space for many years, so thanks for that, Gary. As both of us know, being in the requirement space for a while, requirements reuse is an extremely important concept. Whether you’re creating a common library set of requirements or you’re doing variant development, this is something that we experienced through many years of working with requirements tools. Very important concept, and curious as your perception working with other requirements tools regarding reuse and in particular, maybe the DOORS family of products. How has that been for you?

Gary Hayes: Yeah, it’s been an interesting journey. In the early days of Requisite Pro and actually, my first exposure to requirements tools were with Technology Builders and Calibre. Calibre version 1 as a matter of fact. Back in those days, they were client server tools that really didn’t have any kind of reuse features except copy and paste. Clone and own, if you will. And DOORS, my first exposure to DOORS was that same way. You could create linkages, you could copy modules and reuse those and different projects as they got spun up, but the real reuse didn’t come into play until you got to more modernly architected tools like DOORS Next Generation. DOORS Next Generation has a variety of ways in which they could reuse components or reuse different artifacts with the environment. For one, you could start off a project, and clone an existing project, and use everything that you had before.

But what they really started to do was use the change sets, which was really change sets with code development where you could branch and merge. You really started seeing a lot more sophisticated ways to do reuse within a project. It made it really interesting and it is fairly easy to understand for people using requirements. You really wanted to have a use case that matched up with one, what you needed in an environment. And then also, you wanted to match up with the maturity of your organization. You didn’t want to overwhelm them with a process that they couldn’t handle. DOORS Next Generation took that to the new level by Introducing Global Configuration Management or GCM for short. Very complex way to do business. And it was really a way to include not just requirements for reuse, but all of the artifacts across the software and systems engineering lifecycle.

Really interesting, sounds really great, but like I said, very complex, and once you turned it on to use in your environment, you couldn’t turn it off, so it did not lend itself to a lot of flexibility. It was flexible from the point of view that yeah, I can make components in different disciplines and mix and match them as I chose to, but you really had to have a mature organization and a mature administration group to keep it under control, and make sure everything stayed on track.


RELATED: The Benefits of Jama Connect®: Supercharge Your Systems Development and Engineering Process


Mario Maldari: Yeah, the complexity, I think it’s a challenge for adoption. And I think targeting some of the very large, big customers and modeling their use case, I think that becomes very difficult for smaller customers when they try to use some simple use cases and take them forward. So, I think that that complexity is a challenge. Well, let me show you something. I want to talk a little bit about how reuse is done in Jama, and I’d like to show it to you. Some of the common reuse scenarios that I’ve seen here at Jama, but as well as an industry is developing common libraries of requirements where you’re wanting to develop once and be able to reuse these requirements across the board in different projects, even within the same project. Parallel development. Very common to reuse requirements for your parallel development as well as variant reuse, being able to use them across your variants. These are very common scenarios that we see in industry today.

Jama has a very simple implementation for reuse, but it’s quite powerful. Anything in Jama can be reused, whether it’s a set of requirements or an individual requirement. And to do so, you simply will click on the requirement, and you can say reuse item. And here, you can share this requirement. Within the same project, you can reuse it, or you can reuse it in a different project. And you have a few different options here as well. You can add a relationship from the original item, so you have a link back to it. You can include all tags, attachments, and links. You also have the ability to include the relationships from the source item, include related items as well in minor relationships. A lot of different options when you go to reuse the requirement. And once it’s reused, you can take a look at the requirement itself, and see where it’s been reused. You can see in this case, the current item I have here I’m looking at, but then this requirement’s also shared across and reused in two different projects.

And you’ll see it’s out of sync. That means the requirement’s been evolving and changing in these different projects, which is what you’d expect in this case. Now, if I wanted to get a little bit more information and see okay, well, how are those requirements evolving and changing? I can take a look at the synced items here across the whole project. And if I want to take a look at this particular requirement and see how it’s been evolving and changing, I can take a look at it and I say, “It’s out of sync.” And I can say, “Well, let’s compare.” And here, I can get a side by side comparison of how the requirements he has evolved in this project. You can see the source project, I have the name with a global impact. And in the project that I’ve reused it in, I can see easily that this requirement has changed to a North America scope only. A really nice side-by-side comparison in terms of how the requirements are evolving.

Even more to that, there’s a nice UI here where if I decide that the requirement, that’s evolving, I want to override it with the source, I can do that easily. Or perhaps this requirement that’s evolving should be the new standard. I can overwrite the original with the evolved requirement. A lot of options in terms of managing your reused requirements. But I think the key for me, from my perspective with the Jama implementation, it’s very simple, very easy to use. You can build from a very simple case to a very complex case as you go incrementally, so you’re not overwhelmed instantly with the reuse scenario itself. A nice supporting UI to deal with reuse within Jama. Let me just stop there, Gary, and see if you had any perceptions. You’re relatively new to Jama, so just curious to your thoughts.


RELATED: Eight Ways Requirements Management Software Will Save You Significant Money


Gary Hayes: Yeah, what I really like about this is that it’s easy to enable. It’s really kind of straightforward. It has a variety of use cases that it supports. Got some basic features, but it also supports some advanced features, so you can turn on as much or as little as you need to be effective. Also, the people that are part of the project, the common users, they get that visual cue. I noticed that in the interface, anything that was in a reuse state had the little dot next to it. If I was curious about the reused state of it, I could drill down on that and do some comparisons myself to see how it evolved over time. I think that one, it’s important that it’s easy to use and people aren’t afraid of using it. They can investigate it and it’s very simple. Simple, yet powerful in my estimation.

Mario Maldari: Yeah, I think that’s a good way of saying it. And of course, you and I have been in requirements for 20 years plus, we know that reuse is such an important concept, but it’s really about striking the balance between features and functionality and ease of use. It’s something that every requirement tool needs to have and needs to support. But the question is how easy is it to adopt? And how easy is it to use? You want to find yourself being productive and not wasting a lot of time and energy out of the box.

Gary Hayes: Exactly. Yep, absolutely.

Mario Maldari: Well, Gary, I want to thank you very much for your time today, and want to thank everyone watching this vlog series, and look forward to seeing you on the next one.

Gary Hayes: Thank you.


Is your data working for you? A consistent and scalable data model is instrumental for achieving Live Traceability™ and making data readily available across the development lifecycle.

Download our Jama Software® Data Model Diagnostic to learn more!


Thank you for watching our Episode 8, Jama Connect vs. IBM DOORS: Reuse and Variant Management. To watch other episodes in this series, click HERE.

To learn more about available features in Jama Connect, visit: Empower Your Team and Improve Your Requirements Management Process

We hope you’ll join us for future Jama Connect Jama Connect vs. DOORS topics, including Requirements-Driven Testing and Total Cost of Ownership.



Revolutionizing the Supply Chain: Developments in the Warehouse Robotics Industry
Jama Software is always looking for news that would benefit and inform our industry partners. As such, we’ve curated a series of customer and industry spotlight articles that we found insightful. In this blog post, we share an article, sourced from Data Science Central, titled “Revolutionizing the Supply Chain: Developments in the Warehouse Robotics Industry” – originally authored by Nikita Godse on January 24, 2023.


Revolutionizing the Supply Chain: Developments in the Warehouse Robotics Industry

Warehouse robotics is witnessing steady growth, driven by the increasing adoption of automated solutions in storage for food and beverages, consumer goods, retail, and third-party logistics. The collaboration between the e-commerce sector and warehouse robotics is also a major driver of this market, as it allows for developing increasingly sophisticated warehouse automation systems. Additionally, the advent of autonomous mobile robots (AMRs) and the rising popularity of automated guided vehicles (AGVs) are also fueling the growth of this market.

Reforming Warehouse Operations: Increasing Adoption of AMRs and AGVs

One of the most notable trends in the warehouse robotics market is the increasing adoption of AMRs and AGVs. These robots are allowing for the development of more efficient and cost-effective warehouse operations, as they can automate tasks such as picking, packing, and transporting goods. Additionally, the use of AMRs and AGVs is also allowing for the development of more flexible and scalable warehouse operations, which can better meet the changing needs of businesses.

Efficient and Cost-effective: Rising Collaboration between eCommerce Sector and Warehouse Robotics

Another major trend in the warehouse robotics market is the increasing collaboration between the e-commerce sector and warehouse robotics. This is being driven by the growing demand for efficient and cost-effective warehouse operations, as e-commerce businesses look to keep pace with the rapid growth in online sales. Additionally, the use of warehouse robotics is also allowing e-commerce businesses to better meet the changing needs of their customers by providing faster and more accurate delivery of goods.

 


RELATED: Managing Functional Safety Development Efforts for Robotics Development


Advanced Automation: Increasing Preference for Advanced Robots in Warehouses

As the warehouse robotics market continues to evolve, it is clear that there will be a growing demand for advanced robots in warehouses. This is being driven by the surge in the adoption of robots during the COVID-19 outbreak, as businesses look to automate their operations and reduce their dependence on human labor. Additionally, the increasing adoption of industrial automation and control solutions is also contributing to the growth of this market.

Global Perspective: North America to Lead, Europe to hold Second Largest Share

The warehouse robotics market is a global industry, and it is clear that different regions are at different stages of development. Currently, North America is the largest market for warehouse robotics, driven by the rapid adoption of warehouse automation and the presence of major players in the market such as the United States. However, Europe is expected to hold the second-largest market share, driven by the rising adoption of industrial automation and control solutions in the region.

 


RELATED: 2023 Predictions for Industrial and Consumer Electronics Product Development


Innovations in Warehouse Automation: A Look at Leading Players’ Latest Developments

The warehouse robotics market is a highly competitive industry with several leading players. Some of the major players in the market include KUKA AG, FANUC Corporation, ABB, and Yaskawa Electric Corporation. These companies have made significant investments in the development of warehouse automation solutions and have a strong presence in the market.

Additionally, there are also several startups and smaller companies that are emerging in the market, such as Locus Robotics and GreyOrange, which are focusing on specific areas of warehouse automation. The competitive landscape is expected to become more intense with the entry of new players and increasing investments in the development of warehouse automation solutions.

In the warehouse robotics market, major players such as Kuka AG, FANUC Corporation, and ABB Limited have been focusing on developing advanced robots for warehouse automation. Recently, Kuka AG announced the launch of its new autonomous mobile robot, LBR iiwa, which is designed for use in the logistics and manufacturing sectors. Similarly, FANUC Corporation has introduced its new autonomous mobile robot, CRX-10iA, which is designed for use in the logistics and distribution sectors.

ABB Limited has also recently launched its new autonomous mobile robot, YuMi, which is designed for use in small parts assembly and other applications in the manufacturing sector. These advancements by leading players demonstrate the increasing focus on developing advanced warehouse automation solutions in the market.



An Introduction to Research Use Only (RUO)

In this blog, we recap our eBook, “An Introduction to Research Use Only (RUO)” – Click HERE to download the entire publication.


An Introduction to Research Use Only (RUO)

Learn how it differs from adjacent labels, the FDA and EU guidance, its appropriate use, and the consequences of mislabeling products RUO.

Introduction

In the complex world of medical device development, regulation, and distribution, finding the appropriate label to put on a device may not be simple. When is one label appropriate over another?
Does a device need to go through additional testing, verification, or validation? And what are the consequences of using the wrong label? In this eBook, we’ll cover the differences between Research
Use Only (RUO) and a medical device – although, it’s generally a very clear distinction.

Using the right language and label is critical to complying with best practices. This is why Regulatory Affairs works with the regulatory bodies to ensure that the limitations of the product are properly documented. In a rush to get products to market, it may be tempting to use a Research Use Only (RUO) label to avoid additional regulatory processes while still empowering other researchers and developers. However, there are risks to using the RUO label inappropriately that can have serious consequences for developers, users, and patients. In fact, mislabeling a product is illegal, and punishable. You can see an example warning letter the FDA sent to Carolina Liquid Chemistries Corp after finding intentional mislabeling in 2019 here.

This introduction will provide an overview of the Research Use Only label, how it differs from similar, adjacent labels, its appropriate use, and the consequences of mislabeling products RUO.

What is Research Use Only (RUO)?

The label Research Use Only (RUO) is generally used to indicate products that are intended for scientific research only. They cannot be used for diagnostic or medical purposes. However, there is no standard definition of “research use only,” and the label has slightly different meanings in the European Union and the United States. With the IVDR regulations, RUO products that are being used in
the LDT space are going to be revisited and potentially reclassified as a medical device. With this new classification, teams will likely need to follow design controls, best practices, and industry standards.

What is the FDA guidance on Research Use Only products?

Under the FDA’s guidance issued in 2013, a product labeled Research Use Only is an In Vitro Diagnostic (IVD) product “that is in the laboratory research phase of development and is being shipped
or delivered for an investigation that is not subject to part 812.” The agency includes in this category:

  • “Tests that are in development to identify test kit methodology, necessary components, and analytes to be measured.
  • “Instrumentation, software, or other electrical/mechanical components under development to determine correct settings, subcomponents, subassemblies, basic operational characteristics, and possible use methods.
  • “Reagents under development to determine production methods, purification levels, packaging needs, shelf life, storage conditions, etc.”

The European guidance document MEDDEV 2.14/2 states that a product categorized as an RUO product “must have no intended medical purpose or objective.” The guidance does exempt some tests developed for in-house use as long as the products are not sold to other companies. Some examples of items that can be classified as “research use only” under this exemption include PCR enzymes, gel
component agars, and primers.


RELATED: FDA released new draft guidance of premarket submissions for medical devices – are you ready?


What is the difference between RUO and IVD?

An IVD is an “In Vitro Diagnostic Medical Device,” and the general term applies to any device or product that either alone or with other products is intended to be used for diagnostic, monitoring, or compatibility purposes. There are four different regulatory levels for IVDs:

  • Research Use Only (RUO)
  • General Laboratory Use (GLU)
  • For Performance Studies Only (PSO)
  • In Vitro Diagnostic Medical Device (IVD)

Chart

The simplest explanation for these different levels is that each increasing level requires more testing and oversight. Research Use Only products are at the lowest level of regulation, and In Vitro Diagnostic Medical Devices are at the highest level. Occasionally in the US, products will be labeled as “RUO IVD,” which means an in vitro device that is intended for research use only.

Products labeled with the “CE-IVD” label indicate that they have progressed through the applicable regulatory process and standards (such as IVDD or IVDR). These products are approved for diagnostic use and must include the IVD symbol to be used for medical purposes.

In the EU, as of May 2022, IVDs must comply with Regulation (EU) 2017/746 (IVDR). The IVDR defines IVDs as follows:

“‘in vitro diagnostic medical device’ means any medical device which is a reagent, reagent product, calibrator,
control material, kit, instrument, apparatus, piece of equipment, software or system, whether used alone or in
combination, intended by the manufacturer to be used in vitro for the examination of specimens, including blood
and tissue donations, derived from the human body, solely or principally for the purpose of providing information
on one or more of the following:

(a) concerning a physiological or pathological process or state;
(b) concerning congenital physical or mental impairments;
(c) concerning the predisposition to a medical condition or a disease;
(d) to determine the safety and compatibility with potential recipients;
(e) to predict treatment response or reactions;
(f) to define or monitoring therapeutic measures.”

All IVDs that comply with the IVDR must carry the CE Mark if marketed in the EU.

Research Use Only products are not subject to regulatory requirements in either the US or the EU, but because they don’t meet the same compliance standards as IVDs, they must be clearly labeled as RUO products and cannot be used for medical purposes.

A known exception is the lab developed test (LDT) pathway for clinical purposes.

What are the requirements for an RUO product?

In the US, RUO products are basically unregulated and do not need to meet any specific requirements to carry the RUO label. The FDA does not specify any restrictions or limitations on RUO products, provided they are clearly labeled “For Research Use Only. Not for use in diagnostic procedures.” For this reason, RUO products can be an excellent solution for laboratories that need research materials for testing and research purposes. Because products with the RUO label do not require extensive testing, verification, and validation, they tend to be more cost-effective for research purposes.

The EU rules are similar. Because RUO products do not have clinical applications, they are not considered medical devices, and there are no requirements for RUO products defined by either the IVDD or the IVDR. These products should not be marked with the IVD mark, and they should be clearly labeled as “Research Use Only.”


RELATED: See how Jama Software® helped Össur improve the mobility of millions by replacing process rigidity with speed and agility.


Are there alternatives to RUO labels?

Given the significant differences between labeling a product as RUO and labeling a product as IVD, manufacturers and users can’t be too careful when it comes to assigning labels or using products for
specific purposes. If there is a risk to using products labeled as RUO, manufacturers and users should opt for products that have attained a higher compliance level. For example, for a doctor’s office or home use, IVD is the right path. For clinical purposes or hospital labs, RUO could be used as LDT as long as they are CAP/CLIA certified, such was the case with COVID-19 testing kits when the pandemic first hit.

For products that meet a higher degree of compliance, it is possible to assign General Laboratory Use (GLU), Performance Studies Only (PSO), or even In Vitro Diagnostic Medical Device (IVD) labels. However, depending on the intended use for the Research Use Only products, pursuing these additional levels of compliance may or may not make sense.

What is CLIA certification?

CLIA stands for Clinical Laboratory Improvement Amendments. The Centers for Medicare & Medicaid Services (CMS) regulates all clinical laboratory testing performed on humans in the United States
through CLIA.

What is a CAP accreditation?

CAP stands for The College of American Pathologists (CAP). The purpose of CAP laboratory accreditation is to ensure laboratories provide precise test results for accurate patient diagnoses, meet CLIA and CAP requirements, and demonstrate compliance with professionally and scientifically sound and approved laboratory operating standards.

What are RUO products used for?

As the name implies, RUO projects should be used for research purposes only. They may be used for basic research, pharmaceutical research, or in-house manufacturing of “home brew kits” for research purposes and potentially for clinical applications via the LDT pathway. RUO products are specifically not to be used to make diagnoses, conduct performance studies, or as a substitute or comparator for a CE-IVD device. They may also not be used for market or feasibility studies. Raw ingredients labeled as RUO products may not be incorporated into a finished IVD product.

Learn more about the advantages and disadvantages of the RUO label (and more) by downloading the entire eBook HERE.



DOOR Vlog Episode 7

Jama Connect® vs. IBM®DOORS®: Industry Templates: A User Experience Roundtable Chat

Increasing industry challenges and complexities are pushing innovative organizations to consider modernizing the tool(s) they use for requirements management (RM). In this blog series, Jama Connect® vs. IBM® DOORS®: A User Experience Roundtable Chat, we’ll present several information-packed video blogs covering the challenges that teams face in their project management process.

In Episode 7 of our Roundtable Chat series, Cary BryczekDirector of Solutions Architecture at Jama Software®– and Danny BeerensSenior Consultant at Jama Software®– discuss the importance of industry templates in requirements management.

To watch other episodes in this series, click HERE.

Watch the full video and find the video transcript below to learn more!


VIDEO TRANSCRIPT:

Cary Bryczek: Welcome to part seven of our vlog series. I hope you’re enjoying the series so far. My name is Cary Bryczek, and I’m from Jama Software. I’ve been working here at Jama for over nine years, and I’m the director of aerospace and defense solutions. You’re in for a really special treat today. And I’m excited to be joined by my colleague Danny Beerens, who is a longtime IBM expert. Danny, tell the audience a little bit about yourself.

Danny Beerens: Hi, Cary. Thanks for this introduction. Yes, my name is Danny Beerens. And within Jama, I am both a solution architect for pre-sales and a senior consultant for post-sales. So I’m not going to sell you anything I can’t deliver myself. I’ve been in the requirements management field for the last 15 years, starting with DOORS Telelogic, and after the acquisition by IBM, I moved to IBM Jazz, the ELM Suite, and I actually started out with implementing Rational Requirements Composer. It was the prelude to DOORS Next.

Cary Bryczek: Oh wow. That’s awesome. So today’s topic of industry templates, I think, is a really interesting one. Requirements tools have been around for decades and were originally designed to be just a database. They had some rudimentary capability to control versions and add attributes, and that was about it. There were very few concepts, and still today there are very few concepts of a pre-configured template that were specific to the processes of a particular industry such as automotive or medical devices. When there are no templates and organizations start adopting a new tool, they’re forced to then configure the requirements management system to match the processes to be able to generate reporting information and documentation. Now, this just takes a lot of time, and it forces organizations, in many cases, to have teams of workers whose job it was to just engineer and maintain the tool itself. Danny, what’s been your experience with IBM tools with regards to templates?

Danny Beerens: Well, in general, IBM does offer some industry templates, what they call solution process assets. And for those, there is not a lot of templates available. So you have system software engineering templates. You have a template to support DO 178B and ASPICE ISO 26262.

Cary Bryczek: So in your experience in using DOORS, is there even the concept of creating a template? Is that hard?

Danny Beerens: Yeah, you can create your own templates, but yes, like you mentioned in your introduction, it takes a lot of time and effort from your own organization to implement your own templates to support your own industry. So setting up everything to get ready to start using the application, it takes a lot of effort for your own organization to be set up.


RELATED: ASPICE 101: What is Automotive SPICE?


Cary Bryczek: Oh gosh. Well, it’s so easy in Jama. I think maybe I should let the tool talk for itself. Our templates come right out of the box pre-configured. Let me show you maybe what the building blocks look like. In Jama, totally web-based, our pre-configured industry templates come with item types that represent the nomenclature very specific to the industry. So in A&D and space defense, you’re looking at different types of element requirements, different failure analyses, functional requirements, high-level and low-level requirements for avionic systems, parts. All of these are pre-configured with the most common types of attributes ready for you to use right out of the box. And then where we start talking about how to interconnect all of that kind of data is where we have, in Jama, our relationship rules. And our relationship rules, we have relationship rules for the NASA product breakdown structure.

So if you are following the NASA systems engineering handbook and you’re going all the way from stakeholder expectations to component level requirements, we have this pre-configured for you with all of the correct relationship types. We have one for avionics development that matches all of the DO 178 and ARP 4754. We have templates for MBSE. We have templates for defense system V. We have templates for automotive as well. So all of those are basic automotive framework. If you’re having to do ASPICE or ISO 26262 or both of them together, we have these pre-configured data models for you to start just creating and authoring the requirements right away.

If you’re doing automotive development within the semiconductor industry, we have a template that is developed by industry experts. So we at Jama have hired people that have worked in these industries and know what are the types of compliance reporting that you have to do and then put together these just easy and ready-to-go templates that allow you to get started right away. If you need to do functional safety in automotive or cybersecurity, our templates are built right in, letting you do the safety and cybersecurity right away along with your design development of your requirements. Danny, doesn’t that look like it’ll save organizations time to get started and eliminate the army of DOORS admins?

Danny Beerens: Oh, definitely. If I look at that, your type system is already there. Your relationships are already defined. It saves a world of work there.

Cary Bryczek: So what, Danny, about DOORS NG? What’s the state of industry templates with that tool?

Danny Beerens: If you look at the entire IBM ELM suite, the main focus nowadays is automotive, ASPICE ISO 26262, cybersecurity. And although there are many other multiple templates for different industries like aerospace, medical devices, rail end, they mostly seem to focus on EWM or RTC, as it was called previously. So there is hardly any good templates to support industries endorsing NextGen.

There is an additional application that would allow you to have specific templates or to configure your templates and migrate those to DOORS Next. It’s called Method Composer, but then you also need to know how Method Composer works because you get your templates out of the box. You need to adapt those templates in Method Composer to suit your business processes, and then from there, export it to DOORS Next. So you need an additional application. You need to know how that additional application works to customize it to your processes. Then you need to make sure that DOORS Next gets those templates.

So you need another application, additional infrastructure, and there’s a lot of knowledge in Method Composer. So you need either an IBM expert like I was, an IBM business partner to help you out with that. And in my experience, there is hardly any company that also acquires Method Composer. So you are stuck with setting up your templates in DOORS Next Gen yourself, again.

Cary Bryczek: Wow. God, that just sounds so complicated. So there really aren’t any really ready-to-go templates made for an industry. You still have to do a lot of work.

Danny Beerens: You need a lot of hand work yourself, so you take time away from your own engineering team, from your own process engineers implementing everything in DOORS Next Gen or, like I mentioned previously, in traditional DOORS. So it takes a lot of time away and costs a lot of effort to get you started.

Cary Bryczek: Wow. Is there any process documentation that the end users would use? Anything like that? Or you just have to make that yourself too?


RELATED: Eight Ways Requirements Management Software Will Save You Significant Money


Danny Beerens: Well, you look at your own process or the customer’s process documents, and you try to translate that into a configuration of DOORS or DOORS Next Gen. So there’s really not an easy way to set things up for your industry.

Cary Bryczek: Jama Software, we’ve hired, like I said before, industry experts to put together, not just the templates themselves, but also put together the process manuals that guide users to follow their required industry standards such as the automotive ASPICE and ARP 4754 or IC 6304. Maybe we can take a look at and look at that from a user’s perspective. Let me share my screen here.

Our templates are not just pre-configured in Jama itself, which is very nice. It also comes with a user guide. How do you use Jama itself to follow? This one is a airborne systems process guide. How do I make sure that I really am following ARP 4754 and DL 178? How do I use the system from a high-level process standpoint? So we have these process guides that make it really easy. And then from a user’s perspective, I jump in, I really like the automotive project, we have fantastic sample projects that you can look at, as well as a bare bones skeleton template that you can just start filling in. We have places where you can implement your planning documents in addition to the requirements. We have the dashboards that show you processes that are taking place within your requirements. It’s very easy to follow from the user’s perspective at all.

So I can see what are the different types of requirements, what are the different types of validation testing, or what are the risks that you’re following, and even how the particular system that you are engineering is subdivided from a systems engineering standpoint. So if I want to define the mechanical engineering or the software, I can dig right down in. And this configuration of how the information is organized, how the information is related to one another, it’s all part of the solution, so users don’t have to spend time studying the standards that you have to adhere to or the organization’s process. That’s already built for you. You can just start using it right away, which I think is really cool and a huge time saver and what really differentiates Jama Software from IBM itself.

Danny Beerens: And if you look at your type system and then the documentation that comes along, I must say in my 15 years, I’ve learned that having industry templates is a really great stepping stone to getting a customer quick up and running and achieving their industry compliancy goals. They don’t have to figure it out themselves. It’s already there. You will have guiding documents that explain to you why those templates are set up like they are, what the purpose is, and it comes fully guided with Jama experts. They are experts in the industry, so they can also talk you through the templates if you have any questions. It just simply helps you adjust the framework that comes out of the box to your specific situation within your industry. So you’re already there. You just need to apply rough tweaks to make it your own, and that’s it. And that’s what I love about Jama, working for Jama. How about you?

Cary Bryczek: I love that too. If I’m an end user, I just install Jama, tweak it a little bit here and there and give a minimal amount of training to my users, and then they’re ready to use it. And working at Jama in the aerospace and defense, I really enjoy putting together these solutions and seeing the companies not have to work so hard at using the tool, but actually spending more time innovating on the products and the systems that they’re building. Thanks so much, Danny, for your perspective on the IBM side. That was really insightful. I learned a lot.

Danny Beerens: You’re welcome.

Cary Bryczek: This concludes our vlog on the industry templates and its significance within the requirements management domain. We truly hope you’ve been enjoying the series so far. Stay tuned for our next entry in our series. We look forward to seeing you then.

Danny Beerens: Thanks for having me.

Cary Bryczek: See you next time.


Is your data working for you? A consistent and scalable data model is instrumental for achieving Live Traceability™ and making data readily available across the development lifecycle.

Download our Jama Software® Data Model Diagnostic to learn more!


Thank you for watching our Episode 7, Jama Connect vs. IBM DOORS: Industry Templates. To watch other episodes in this series, click HERE.

To learn more about available features in Jama Connect, visit: Empower Your Team and Improve Your Requirements Management Process

We hope you’ll join us for future Jama Connect Jama Connect vs. DOORS topics, including: Industry Templates; Reuse and Variant Management; Requirements-Driven Testing; Total Cost of Ownership; and Why Did We Move to Jama Connect? A Customer’s Story.

Understanding Integrated Risk Management for Medical Devices

In this blog, we’ll recap our whitepaper, “Understanding Integrated Risk Management for Medical Devices” – To read the entire paper, click HERE.


Understanding Integrated Risk Management for Medical Devices

Knowledge on best practices, how to integrate risk-based thinking into product development cycles, and the importance of having end-to-end traceability to improve risk management, shared by industry and solution experts.

A level of risk exists with all medical devices, no matter how simple they are.Companies developing medical devices are constantly considering who (or what environment, facility, etc.) could potentially be hurt by a device so they can help reduce risk and meet regulatory requirements. Risk management in the context of ISO 14971 is designed to support medical device manufacturers with these tasks — but not all approaches are equal.

The amount of time it takes to manage risks, connect specific risks to specific requirement tasks, and pull together required documents to respond to an audit varies slightly depending on the approach. The risk management process is an integrated process that not only includes teams in product development, quality, but also many other parts of an organization.

This whitepaper taps into the knowledge of industry and solution experts to uncover best practices, how to integrate risk-based thinking into product development cycles, and the importance of having end-to end traceability to improve risk management. Before we dig into integrated risk management, let’s first define some key terms.


RELATED: Jama Connect® Features in Five: Risk Management for Medical Device


Risk Management Terms According to ISO 14971

Harm – Harm occurs when people are injured physically or their health is compromised or when property or the environment is damaged.

Hazard – A hazard is a potential source of harm. Annex E.2 categorizes hazards in the following way: energy hazards, chemical hazards, biological hazards, operationalhazards, and informational hazards.

Hazardous – A hazardous situation occurs when people are exposed to a hazard or when property or the environment is threatened. A hazardous situation exists when a vulnerable entity is exposed to a hazard.

Situation – According to ISO 14971, the concept of risk combines two variables: the probability of harm and the severity of harm.

Risk – For example, if a particular hazardous situation is very likely to cause harm and would be very harmful if it actually occurred, then it would be a high risk situation. Conversely, if it’s very unlikely to cause harm and would be only slightly harmful if it actually occurred, then it would be a trivial risk.

Risk Analysis – Risk analysis is a systematic process that is used to identify hazards and to estimate risk. It includes an examination of every reasonably foreseeable sequence or combination of events that could produce a hazardous situation and cause harm.

Risk Assessment – Risk assessment is a process that is, in turn, made up of two interconnected processes: risk analysis and risk evaluation.

Risk Evaluation – Risk evaluation is a process that is used to examine the estimated risk for each hazardous situation and then to use risk acceptability criteria to determine whether
or not the estimated risk is acceptable and to decide if risk reduction is required.

Risk Control – Risk control is a process that is used to consider risk control options and to select and implement risk control measures that will reduce risk or maintain risk within
specified levels. ISO 14971 expects you to consider the following risk control options and, if possible, to apply them in the following order:

  1. Design safety into the product.
  2. Establish protective measures.
  3. Provide safety information.

Risk Estimation – Risk estimation is a process that is used to assign qualitative or quantitative probability values and severity values to each hazardous situation. These values are then used to estimate risk.

Risk Management – Risk management uses policies, procedures, and practices to systematically analyze, evaluate, control, and monitor risk.

Safety – Safety is freedom from unacceptable risk. Risk acceptability criteria are used to help decide whether or not a risk is unacceptable.

Severity – Severity is a measure of the possible harmful consequences that a hazard could potentially cause.


RELATED: Download our whitepaper, Application of Risk Analysis
Techniques in Jama Connect® to Satisfy ISO 14971


The Risk Management Process

During risk management — after one defines a device’s intended use(s) — risk analysis can begin with identifying all potential hazards, and hazardous situations. Once this is defined, risk can be estimated and can determine the type of appropriate risk control required. Once the risk controls are implemented, residual risk needs to be analyzed to ensure that the benefits outweigh the risks. Let’s take a look at what’s involved in the risk management process.

Identifying Hazards

“Risk” is defined as the severity and probability that harm will occur. Defining the severity of harm requires you to identify all the known and foreseeable hazards for both intended and unintended uses.

For example, let’s say you have an infusion pump, and that pump has air in the line, which creates a hazardous situation for the patient. Different levels of patient harm can occur, so it’s about uncovering the possible scenarios and the likelihood of a situation’s occurring.

Risk Harm

Understanding Harm

Understanding harm includes both people and property. A medical device that catches fire might threaten property, while an infusion pump with air in the line might threaten human life. Think about what could cause harm to people, like a shark swimming in the water. A shark that attacks a person could create different levels of harm. A few examples include loss of a limb, an infection from getting bitten and loss of life. The various levels of harm result from the hazardous situation, which is the shark in the water.

Harm Severity

Risk Evaluation

Risk evaluation involves comparing an estimated risk against a specific criterion to determine if a risk is acceptable. Five different levels to evaluate risk are common practice, but you can use as many as you’d like. The most severe risk (level five) might include death or impairment. Level one might include no risk to a patient or operator. The levels inbetween include all the other varying
degrees of risk.

Sequence of Events

A hazardous event includes a number of steps, which is the sequence of events. A risk situation might have two, three, or more steps that, when aligned, create a hazardous event. Risk management tools such as fault trees and failure modes and effects analysis (FMEA) help identify these steps.

Previous version of ISO 14971 used terms like “acceptable” and “unacceptable” to describe risks, but that language has since been removed and the most current version maintains as low as possible (ALAP). The goal of every manufacturer is to lower the risk as much as possible and rethinking how to prioritize risk controls can help.

Harm Flow Chart

This has been a preview of the content in our whitepaper, Understanding Integrated Risk Management for Medical Devices to read the entire paper, click HERE

 

Jama Connect® Features in Five: Risk Management for Medical Device

Learn how you can supercharge your systems development process! In this blog series, we’re pulling back the curtains to give you a look at a few of Jama Connect®’s powerful features… in under five minutes.

We always want to be respectful of your valuable time, but in this Features in Five video, we do go beyond the promised five-minute format to include an information-packed session. Join Vincent Balgos, Director of Medical Solutions at Jama Software®, as he walks through how risk management is integrated into the Jama Connect for Medical Device framework and how our new Lookup Matrix feature can fit seamlessly with your organization’s risk processes

In this session, viewers will learn how:

  • Jama Connect for Medical Device framework helps organizations align with regulations such as ISO 13485:2016, 21 CFR 820.30, and ISO 14971:2019
  • How to set up and utilize our new Lookup Matrix feature

Follow along with this short video below to learn more – and find the full video transcript below!


VIDEO TRANSCRIPT:

Vincent Balgos: Hi, my name is Vincent Balgos, and I’m the director of Medical Solutions here at Jama Software. In this video, I’m going to provide an overview of how Risk Management is integrated as part of our medical device framework offering. In addition, I like to show you a new feature called Lookup Matrix that can fit seamlessly with your risk processes in a few easy steps.

So when developing complex products, integrating risk management activities can be a complicated process, usually requiring experts from various functional groups. Without a connected tool, there’s a potential for fragmented risk activities across the different siloed groups. This may lead to an impact on the product, its function, its safety, and its users.
In Jama Connects out of the box medical device framework, the risk management process is already integrated with other elements of key product development. These practices are aligned with ISO standards such as 13485, design controls 820.30, and also, of course, 14971. Seeing the relationship diagram of the medical device framework that’s readily available out of the box, hazards are a standalone item type that allows users to identify hazards that come directly from the product’s intended use, which is compliant with ISO 14971 section 5.4.

Related to the hazards is a risk evaluation item type that contains fields that also align with 14971 best practices, such as hazard situation, sequence of events, harm and severity, and probabilities, both pre mitigate and post mitigate. The risk evaluation is in trace downstream to risk control requirements and their verification activities. This continues to comply with the standard of practice as defined in 14971. Let’s dive into risk analysis. I’d like to talk to you about our new Lookup Matrix. Released in 8.75, this new feature allows the use of dedicated pick list inputs to automatically output desired content based on a pre-configured lookup table. The Lookup Matrix offers an easy-to-use interface, which allows for seamless analysis within the tool, which aligns with your organization’s process.


RELATED: Jama Connect® for Medical Device Development Datasheet


Vincent Balgos: As you can see here on the right, here is the general setup process, which I’m actually going to walk through right now. Here’s the out-of-the-box medical device framework. If you go into the admin section, one of the first steps is really to configure the Lookup Matrix. In the admin section, the first thing you have to do is define your pick list. So if I go here on the left, as you can see here, I’ve already set up an example for my Lookup Matrix, but this is an easy task to do to create your own that you want. So, for example, if I need to create a brand new pick list, all I have to do is hit the green button here. We’ll call this new pick list or lookup. And the key thing is here is instead of choosing the standard, we actually need to select the new Lookup Matrix.

And what this does is it’ll create a brand new pick list where then you can go ahead and define the different options as you have in your other standard pick lists. As you can see here, here are some options that, again, taking it straight from 14971, some good examples for probability. Some info tips and some color and stuff like that. So you can see here, I’ve created four different pick lists that will serve as input into my Lookup Matrix. That’s step one.

Step two is then to actually create your Lookup Matrix. Select on Lookup Matrix. You can actually see that here’s a new field where you can configure your Lookup Matrix. I’ve already pre-established this, but if I hit the view button, I can see that my probability P1 and P2 pick list is actually input on both my X and Y axes. And I can actually determine what is that total probability based off these two inputs. The second Lookup Matrix that we need to create is the risk lookup acceptability. Which actually takes input from the probability total Lookup Matrix that we just talked about and then compares it against the severity of it.

From here, this is where then you can do a lookup analysis based off your organization’s risk management process. This is pre-configured, but if you wanted to create your own, it’s as easy as hitting a couple buttons. All you have to do is add a new Lookup Matrix. Now we’ll call this new Lookup Matrix. And then from here I just have to pick what are my input pick list. So for my X axis let’s pick severity, like we did before. And then, for my Y-axis, we’ll go ahead and pick the total probability. As for the values of it, this is where the risk acceptability value is another input Lookup Matrix pick list that we have to create.

Once you’ve established that, you go ahead and hit generate matrix, and then the bottom screen here, you can actually then configure your matrix per your organization. So, for example, here, I can click that this is low, but maybe here, this is where I want to actually then increase the level of risk. As you can see here, this is quite easily configurable and easily managed. Now that you’ve established your Lookup Matrix, the last part is actually configuring your item type fields. So for this particular example, I created a brand new item type called the risk evaluation Lookup Matrix. And one of the key things you have to do is actually identify that this is a calculated logic field and that you’d like to use your Lookup Matrix.


RELATED: Carnegie Mellon University Software Engineering Program Teaches Modern Software Engineering Using Jama Connect®


Vincent Balgos: As you can see here, here’s this has been predefined, but again, and if you wanted to add your own, it’s easy as adding your own field. So, for example, if we go ahead and hit add field, I go to customs, go ahead and hit calculate and logic. And we’ll say this is our new risk lookup. Once I come down to here, I establish that. For my calculation type, I actually just select Lookup Matrix. And then as my calculation source, I actually want to define my risk acceptability. And then here’s where then I define what are the fields to be used as input. So for this one here, it becomes pre-populated with the severity. But the next one is like, do I want to talk about pre-mitigated P-total or residual P-total? Why don’t we go ahead and pick residual. So that can establish your new field. Now that you configured your Lookup Matrix appropriately within your admin section, why don’t we jump to the project and see how that looks.

As you can see in this new project, I created this new item type called a risk evaluation Lookup Matrix. In the list view, as you can see, here are some pre-populated examples of this new item type with the Lookup Matrix functionality enabled. As you can see here, this follows the general standard best practice with 14971 in terms of identifying your hazards, sequence of events, hazardous situation. But more importantly, I identify what the severity is, the P1, P2, and P total. Again, based off your Lookup Matrix configuration, this is predetermined. But I can see here for this example, on my risk example five, that actually I have not identified what is my P1 mitigated. But this is really nice because with this view, you can take a look at seeing what has not been enabled, and then you can address it immediately right here on the screen. Let me show you how.

So for this particular example, I can see that my P1 hasn’t been defined yet. This is easily addressed by just go ahead and double click this. Let’s say with the, this is again, this is pre-mitigated, this is probably a more a probable type of event. I can go ahead and hit probable, and you can see that my “P total” is automatically updated again based off of my first Lookup Matrix. And then my risk level is also then determined based off that second Lookup Matrix that we’ve had here. If I further go down the list here, I can see I can go back and then see, hey, let’s define what our P1 mitigator looks like. So then with that, let’s say I put it in the risk controls and say, this is now improbable. I can go ahead and update based off of that. My “P total” has been reduced now to medium, but my risk level is still high. Again, this shows you can apply dynamic changes and analysis on your risk management activities on the fly.


To learn more about available features in Jama Connect, visit: Jama Connect Features

We hope you’ll join us for future Jama Connect Features in Five topics, including Reviews, Dashboard Management, and more.

In this blog, we partially recap this customer story, “Vave Health Migrates to Jama Connect® to Accelerate Development and FDA Clearance” Read the entire story HERE.


Vave Health is committed to revolutionizing the physician-patient experience through innovative, industry-transforming technologies. Their innovative handheld ultrasound device packs the ability to wirelessly connect with your Android or iOS smartphone or tablet.

After initially selecting Matrix Requirements, Vave Health found themselves constrained by the tool’s limited functionality and were ready for a change. Following a requirements management market analysis, Jama Connect® was selected and onboarded due to its ease of use and industry-leading functionality.

Read this customer story to see how now, with more confidence in their processes, Vave health has achieved the following outcomes:

  • Accelerate the release cadence from what previously took a couple weeks, down to a day or two
  • Decrease generation of trace matrices from 30 days to one per project
  • Scale development process with the ability to run multiple projects in parallel
  • Maintain traceability and instantly identify coverage for verification and validation of requirements to respond to action items sooner in development

VAVE HEALTH CUSTOMER STORY OVERVIEW

CHALLENGES WITH MATRIX

  • Reports, such as a traceability matrix, were taking too long to generate
  • The steep learning curve caused most people to revert to working in Word and Excel
  • Inability to develop parallel projects and reuse data between releases, contributed to duplicated work and slower-than-desired release cadence

SELECTION CRITERIA

  • A solution that would scale with their growth
  • Quick-to-adopt and easy-to-use 
  • Strong market presence
  • Ease of data migration

OUTCOME + FUTURE

  • Accelerate the release cadence from what previously took a couple of weeks, down to a day or two
  • Decrease generation of trace matrices from 30 days to one per project
  • Scale development process with the ability to run multiple projects in parallel
  • Maintain traceability and instantly identify coverage for verification and validation of requirements to respond to action items sooner in development

RELATED: 2023 Predictions for Medical Device Product Development


CHALLENGES

In the early days of Vave Health, the development team originally selected Matrix Requirements due to its low cost. While the tool was sufficient for managing their requirements in the preliminary stages of development, as the company began to scale, it became apparent that they needed a more mature, enterprise-grade solution with more robust capabilities.

The main challenges that Vave Health had which led them to seek out a new solution were:

  • Reports, such as a traceability matrix, were taking too long to generate
  • Steep learning curve caused most people to revert to working in Word and Excel
  • Inability to develop parallel projects and reuse data between releases, contributed to duplicated work and slower-than-desired release cadence

As a small team, they did not have dedicated staff to manage requirements – it was a shared responsibility. With Matrix Requirements, the learning curve was so steep, only a few people were able to use it. Even then, it was used similarly to an Excel spreadsheet.

“Matrix Requirements was difficult to use, and it limited our ability to easily extract reports and quickly show traceability. The whole process just took too long,” said Craig Loomis, Vice President of Product at Vave Health.

“One of the deliverables in getting our product released is generating the trace matrix. With Matrix Requirements, it was very cumbersome,” said Sandhya Mitnala, Head of Quality and Regulatory at Vave Health. “We realized that something that should have taken one or two days, and managed through the project, took us almost a month. It was a very manual process.”

Additionally, as the team grew and the development work went from singular to multiple projects, the team ran into limitations using Matrix Requirements.

“One thing we didn’t initially think about when selecting Matrix Requirements was the ability to have multiple projects in motion at the same time,” said Loomis. “Although it was technically possible, there was no good way to extract the trace matrix and manage revisions across different projects at the same time in parallel.”


RELATED: FDA Updates to the Medical Device Cybersecurity Guidance


SELECTION CONSIDERATIONS

In order to overcome the limitations of their current tool, the team set out to find a solution that could meet their current needs and grow with them as they expanded in the market.

“The startup world is unique in that you’re trying to do so much more with fewer resources. Sometimes you do need to leverage technology to automate things that larger companies would be
able to throw bodies at,” said Loomis.

When it came time to evaluate the available solutions in the marketplace, things moved quickly.

“From my experience working with Jama Software® at other companies, and my coworkers’ similar experiences, we wanted to move to a more automated solution and Jama Software was on everyone’s mind,” said Mitnala. “It was a very easy choice for us. All of the solutions we looked at, outside of Jama Connect®, were ruled out quickly,” shared Loomis.

During the evaluation process, the Vave Health team was able to access a sandbox account created specifically for them, so they could test out the solution to make sure it was the right fit.
Because there were so many things already in motion, the team wanted to ensure that data migration would not be an issue, so they could keep moving quickly.

“Before we even signed a contract, we spent time in Jama Connect and had a lot of confidence in moving forward. We knew that our data would be migrated easily, and we wouldn’t be putting our projects at risk,” said Loomis.

Although Matrix Requirements supported some initial needs, the team knew that in order to derive the value they needed, it was time to up-level their tool for requirements management and systems engineering. The return on investment for Jama Connect, a robust, yet easy-to-use platform (which comprised the feature set and functionality they required) would increase efficiencies, simplify compliance, reduce risk, and ultimately speed time-to-market, paying dividends in the long run.

“As a startup, the one thing you must ensure is that you are able to move fast. You’re learning the market, you’re working against your competitors, and speed to market is critical. Especially where there are things that can be automated – that’s where you want to invest,” said Loomis.

To read the entire outcome from Vave Health’s choice of Jama Connect, read the entire customer story here:
Vave Health Migrates to Jama Connect® to Accelerate Development and FDA Clearance