Tag Archive for: ROI

Alternative to IBM DOORSProduct development teams face many challenges in today’s fast-moving and increasingly regulated environment. Potential missteps, however, can create an expensive ripple effect throughout the product development cycle, with the potential for missed deadlines, compliance issues and more.  

Real-time collaboration and the need for a single source of truth are critical to product development teams. Outdated tools often don’t deliver on these needs, and a misalignment between what a team needs and what a tool provides can hinder success. Requirements management tools that keep pace with your team’s evolving needs can help mitigate potential risks, improve efficiency and achieve faster results.  

If you’re currently using IBM DOORS, understanding the difference between the existing solution and more modern options can help you to determine the best next steps for your organization.  

Requirements Management and IBM DOORS 

Many organizations adopted IBM DOORS because they needed a requirements management tool for their teams and, at the time, it seemed like the best option. But much has changed since the creation of IBM DOORS, and the tool has quickly become outdated. Today’s teams need the ability to not only collaborate in real time but also to do so remotely. They also need to:  

  • Follow a consistent and common requirements practice.  
  • Create a single source of truth for requirements, ensuring that everyone on the team is working from the same information.  
  • Have easy integration so that requirements management is integrated into both core workflows and business to improve productivity.  
  • Be able to track requirements to develop, test and release new products.  

Development teams and critical stakeholders expect the tools they use to be as intuitive as the technology in other areas of their lives. Users of IBM DOORS, however, struggle with usability issues, leading many to not use the solution and opt instead to use outside programs, such as Word or Excel. This creates challenges within workflows, productivity and efficiency, and it introduces potential risks, eliminating the ability to have a single source of truth. But how did IBM DOORS become outdated, and what does it lack?  

RELATED ARTICLE: Is There Life After DOORS®?

The Advantages and Disadvantages of IBM DOORS  

Organizations are often tempted to maintain an outdated solution, even if it’s underperforming. Why? Because fear and worry exist when it comes to switching to a new solution, users often stay with the existing solution or upgrade to the next offering within the same organization (such as IBM DOORS Next).  

The advantage of staying with IBM DOORS is that you don’t need to invest in a new solution and undergo the adoption process that comes with transition. But delaying the switch to a new RM tool only delays the inevitable, because IBM DOORS will eventually go out of support, which will create many challenges, including potential security issues.  

A few common misconceptions around the advantages of IBM DOORS include:  

  • Upgrading to IBM DOORS Next is a less-expensive option. IBM DOORS Next is not IBM DOORS, it’s an entirely new tool with a new approach to requirements. In fact, the only thing the two have in common is in the name. Migrating to IBM DOORS Next might seem like the less-expensive option. However, the work that goes into upgrading to IBM DOORS Next and transitioning to an entirely new RM tool is the same. Moreover, selecting a different solution may help you improve efficiency and achieve ROI faster.  
  • Customizations will carry over to IBM DOORS Next. Organizations invested money and resources in IBM DOORS customizations (DXL), and you might believe that you can take these innovations with you when you move on to IBM DOORS Next. However, this isn’t the case, and selecting a different solution could allow for you to use fewer customizations.  
  • Business disruption is more significant with a different RM solution. Eventually, you’ll need to move away from IBM DOORS after the support is discontinued. The right RM solution will help teams more effectively hit deadlines, collaborate with greater ease and improve business outcomes, offsetting any upfront business transition.  
  • The user experience will suffer. Many people refuse to use IBM DOORS due to a challenging user experience; switching to a new RM tool may accelerate concept, design and validation processes, leading to faster times to market.  

If you’re considering making a switch, comparing various options can help determine which RM tool is best aligned with your needs. An innovative tool has the potential to reduce risk for project rework, expensive errors and compliance risks.  

What is the best alternative to IBM DOORS?  

If you want to move away from IBM DOORS, you aren’t constrained by a specific path to migration. So, what is the best alternative to IBM DOORS?  

The answer is found in examining both what your team needs most right now and what they may need in the future.  

Collaboration is a critical part of the workflow, especially with the increasing number of people working remotely. Organizations need a tool that supports digital transformation, has greater efficiency and is user-friendly enough that people will use it. 

Jama Connect is a modern alternative to traditional legacy platforms, such as IBM DOORS, and was named the overall leader (No. 1) in requirements management software on G2, outranking IBM DOORS Next for implementation, time adoption, ROI and market presence. The tool offers users a reliable solution with:  

Simplicity. Jama Connect provides an intuitive and modern user experience. Requirements management software supports multiple development methodologies and engineering disciplines to drive cross-team collaboration and alignment.  

Flexibility. Customization is a critical factor to product development teams, which ensures that they get the functionality they need most. Jama Connect provides customization, security development and a licensing model that delivers a lower total cost to ownership.  

Open. A poor experience is created when an RM tool doesn’t integrate with other tools that your team wants to use. Jama Connect allows for seamless integration with the most commonly used tools across the product development life cycle. You’ll have access to a powerful network of options to get the right technology stack aligned to your unique business needs.  

Simplicity, flexibility and the ability to easily integrate with other tools give your team the resources they need to create greater success in all of their projects. As you consider adopting RM tools such as Jama Connect, it helps to have a quick comparison to help guide your decision.  

RELATED: Q&A with a Former IBM® DOORS® Evangelist

IBM DOORS vs. Jama Connect — A quick comparison  

Adopting a new RM tool involves asking many questions, including the following: What is the implementation process? What are the costs and ROI? How easy is the tool to use? Consider the following as you weigh the advantages and disadvantages of IBM DOORS Next and Jama Connect.  

Adoption. Organizations worry that adopting a new solution will take too much time and money, so they often gravitate to a solution such as IBM DOORS Next, primarily because they assume adoption is easier. However, Jama Connect is actually 2.7x faster to adopt than IBM DOORS Next. Additionally, Jama Connect rated 80% in ease of administration, compared with IBM DOORS Next, which rated 71%.  

Return on investment. How fast will your investment pay off? This is a critical question for any new solution, and it’s important to note that ROI is achieved 45% faster with Jama Connect compared with IBM DOORS Next.  

Usability. Usability is a key reason many people refuse to use IBM DOORS and instead use email communication, Word, Excel or other outside applications. Users expect their RM experience to be as intuitive as applications in their social environment. Jama Connect rated 85% in “ease of doing business,” compared with IBM DOORS Next, which rated 74%. 

Supports remote working. The remote working trend is only expected to grow in the future. IBM DOORS lacks cloud capabilities, creating challenges with working anywhere, anytime. IBM Jama Connect creates a single repository so it’s easy for remote teams to gather, review and execute on requirements. Structured reviews and collaboration enable teams to elicit feedback, review product features in real-time with stakeholders, and track critical decisions across teams and locations. 

Moving forward with greater confidence   

Products, systems and software development are only getting more complex; so not modernizing your requirements management tool creates potential risks, such as negative outcomes in your product development process.  

As your team increasingly requires the ability to adopt, innovate and grow, continuing to use IBM DOORS will only become more difficult and potentially introduce risk into your product development process.  

Transitioning to a new requirements management tool provides your team with the resources required to innovate, meet deadlines and succeed. You can more effectively define, manage and validate complex system requirements, all while eliminating the risk and inefficiencies associated with outside documents and legacy systems.  

Achieving Live Traceability™ with Jama Connect®

Jama Software®‘s Live Traceability™ allows engineering teams to quickly and easily access the latest and most complete information for any requirement, no matter the stage of development or tools used. This real-time capability boosts productivity by ensuring teams work with the latest data and reduces risks like delays and defects by finding issues early. Research shows that issues found late can be much more expensive to fix, which is why Live Traceability is so important. Jama Connect® helps overcome the limitations of older tools, leading to better results in many industries such as automotive, medical devices, aerospace & defense, and more. To learn more, visit Buyer’s Guide: Selecting a Requirements Management and Traceability Solution

If you’ve just successfully gone through the process of building a business case for a software platform, then you know it’s more than just proving ROI – purchasing software takes a lot of pre-planning and strategy. You’re probably also familiar with the feeling of elation that comes after you’ve received that official purchase approval. 

You also know that once the euphoria wears off, the real work of justifying your purchase begins. That’s because achieving the ultimate value from a solution isn’t always a feat that’s realized within the first year of usage. 

Even the most affordable, desperately needed software will have to prove itself to your decision-makers and fellow employees — not just once, but many times when renewals come due. Building a business case for purchasing software often requires that you keep “selling” the software to your superiors and colleagues, so they maintain a good perception of it and trust you with future decisions. 

How can you help justify purchasing software? Here are five ways.

Trace Everything Back to Your Internal Needs Analysis  

There’s an old saying, “Nobody ever got fired for buying IBM.” Perhaps the modern equivalent is, “Nobody ever got fired for implementing a solution that did exactly what it was supposed to do.” 

Before you began the process of evaluating and purchasing software, you and your colleagues no doubt performed an internal needs analysis that identified the top priorities and anticipated areas of benefit for your new solution. You’d be surprised at how often evaluation teams forget all about this analysis once they’ve gone live. 

So, if you’re building a business case for purchasing software, take the time to recall the main things this solution was supposed to be doing for your organization (Improving efficiency? Streamlining communication? Reducing errors?), and then measure how well it is actually doing these things. You can justify purchasing software in many ways, but if you can tie the value of the solution directly to your company’s internal needs, you’re well on your way to demonstrating ROI. 

Don’t Ignore the Soft Benefits  

ROI is important, but it’s not the only factor to be considered when building a business case. Executives like to see that a $50,000 piece of software paid for itself within one year and continues to deliver savings of $10,000 per year. But don’t overlook opportunities to report on all the “softer” ways the solution is improving life at your company. 

For example, if the software enabled you to streamline a process from five steps to two, you may not always be able to measure the exact time savings, tie that to someone’s salary and come up with monthly or yearly cost savings. But anyone can appreciate shaving steps from a process. 

Of course, sometimes the benefits are even softer. An employee may tell you, for example, that the review process is now “much easier, and even kind of fun.” You can’t say with certainty that the process is reducing turnover by X percent each year, resulting in hard-dollar savings. But it’s intuitive that people will be less likely to leave jobs in which boring or cumbersome transactions are made as painless as possible. 

Learn more about thoughtfully selecting the right product development platform by reading our guide.

Meet with Resistance Head-On 

It’s all too easy to hear negative chatter, shrug, and say “Haters gonna hate.” But in this case, haters are also going to speak ill of your solution to their peers and superiors. And sour grapes from a few disgruntled employees can drag down everyone’s perception of a software platform. 

So, as part of your process for building a business case, meet with the haters. Ask them to explain their concerns about the new system. 

Perhaps they simply need more training to make better use of its features. Or maybe they have legitimate concerns about navigation or functionality that could be addressed through customizations or integrations. 

Watch our webinar to hear more best practices for implementing new technologies. 

Keep Communicating with End Users  

If you’re responsible for purchasing software, make sure your new solution remains a “big deal” long after end user training and the initial go-live date. Even if your launch was a success, you’ll need to encourage employees to keep using the new platform rather than reverting back to their older, less efficient ways of doing things. 

Make sure the communication is a two-way effort. Rather than simply blasting employees with emails reminding them about how great the new software is, consider giving them an intranet page or social media group in which they can share tips, ask questions and even vent frustrations.

Own the Negatives

This one is counterintuitive, but it’s important. There’s no such thing as flawless software, nor even software that’s a perfect fit for any one organization. Your new software has some shortcomings, doesn’t it? Or at the very least, hasn’t it underwhelmed in at least one benefit area? 

Give your decision makers an honest assessment in these areas. When you admit that things aren’t perfect on your new platform, you’ll paint a much more realistic picture of what’s actually going on — and your executives will be more likely to believe you when you gush about the benefits. 

 

Learn more about communicating the benefits of Jama Connect and gaining consensus across your organization by downloading our eBook, “Making a Case for Jama Connect.”  

 

blog-featured-image-ROBO

Overview

Automation remains one of the most contentious topics in software development. You get three engineers into a room to discuss automation and you will end up with four contradicting absolutes and definitions. So for the purpose of this post, we will place the following limits on the topic:

  • Automation in this post will refer specifically to Automation of Functional Tests for a UI or API.
    • Functional Tests will be defined as tests containing a prescribed set of steps executed via an interface connected to consistent data which produces an identical result each time its executed.
  • Failure in this document will be defined as greater than 3 months of effort spent creating automation that is not acted upon or is determined to be too expensive or buggy to maintain after 12 months and is turned off or ignored.

This post will cover the three most common reasons automation fails:

  1. Inability to describe a specific business need/objective that automation can solve.
  2. Automation is treated as a time-boxed activity and not a business/development process.
  3. Automation is created by a collective without a strong owner who is responsible for standards.

Wait, Who are You?

I am Michael Cowan, Senior QA Engineer at Jama Software. Over the past 20 years I have been a tester, developer, engineer, manager and QA architect. I have built automation solutions for Windows, Web, Mobile and APIs. I have orchestrated the creation of truly spectacular monstrosities that have wasted large sums of money/resources as well as designed frameworks that have enabled global teams to work together on different platforms, saving large(r) sums of money.

I have had the amazing opportunity to be the lead designer and implementer of automation for complex systems that handled millions of transactions a day (Comcast), dealt with 20 year old systems managing millions of dollars (banking), worked in high security/zero tolerance (Homeland Security) environments and processed massive big data streams (Facebook partner). I have worked side by side with brilliant people, attended conferences and trainings, as well as given my own talks and lectures.

I have a very “old school” business focused philosophy when it comes to automation. To me it is not a journey or an exploration of cool technology. Automation is a tool to reduce development and operating costs, while freeing up resources to work on more complicated things. I strongly believe that automation is a value add for companies that correctly invest in it, and a time/money sink for companies that let it run wild in their organizations.

Failure Reason #1: Unable to describe a specific business need/objective that automation can solve

The harsh truth is that, by itself, clicking a button on a page (even a really cool/difficult/complex custom button) has no value to the business. The business doesn’t care if you click that button manually with the mouse, execute it with Javascript, call the business logic via API or directly manipulate the database. What they care about is ensuring that a customer is not going to call up after the release to return the product, or some blogger wont discover a major issue and drive away investors with an scathing review.

Automation Projects fail when they are technical exercises that are not tied to specific business needs. If the ROI (Return on Investment) is not clearly understood, you are unlikely to get the funding to do automation correctly. Instead you will find your efforts rushed to just implement ‘Automation’ and move on. Months later, everyone is confused why automation hasn’t been completed, why it doesn’t do x, y, z and why all the things they assumed would be included were never planned.

Nothing is worse than a team of automation engineers thinking they are making great progress, just to have the business decide to pull apart the team due to a lack of understanding the value. If you are running automation directly tied to an understood business need, then the business leaders will be invested. You will find support because your metric will clearly show the value being produced.

Another consequence of running automation as an engineering project is in making decisions based on technology instead of business need. If you decide upfront to use some open source tool you read about, you will find yourself telling the business what it (you) can’t do. Well no, our tool doesn’t hook into our build server, but we can stop writing tests and build a shim. Pretty soon you are spending all your time converting your project to be more feature rich, instead of creating the test cases the business needs. This is how teams can spend 6-12 months building a handful of simple automation scripts. Even worse, you end up with a large code base that now needs to be maintained. The majority of your time will have been spent building shims, hacks and adding complexity that has nothing to do with your business offering or domain.

Mitigation

It’s actually very easy to avoid this pitfall. Don’t start writing tests until you have a plan for what automation will look like when its fully implemented. If your team practices continuous integration (running tests as part of the build), don’t start off with a solution that doesn’t have built in support for your CI/Build system. Find an industry standard tool or technology that meets the business needs, create a POC (Proof of Concept) that proves your proposed solution integrates correctly and can generate the exact metrics the business needs.

Write a single test to showcase running through your system and generating metrics. Make sure the stakeholders take the time to understand the proposed output and that they know what decisions that information would impact. Get a documented agreement before moving forward and then drive everything you do to produce those metrics. If anything in the business changes, start with reevaluating the metrics and resetting expectations. Once everyone is on the same page start working backwards to update the code. Consistency and accuracy in your reports will be worth more to the business than any cool technical solution or breakthrough that you try to explain upwards.

If you are in management, you might consider asking for daily automation results with explanations of all test failures. If the team can not produce that information, have them stop building test cases until the infrastructure is done.

Key deliverables that should be produced before building out automated scripts:

  • A documented business plan/proposal that clearly lays out the SMART goal you are trying to accomplish.
    • Signed off by the most senior owners of technology in your company.
    • This should be tied to their success.
  • Clear measurements for success. E.g. Reduce regression time, increase coverage for critical workflows, etc.
  • The reports and metrics you will need to support the measurement.
    • Your proposal should include a template with sample (fake) data for all reports.
  • A turnkey process generates those reports and metrics from automation results data.
  • 1 single automated test that does a simple test on a real system and generates a business report.

Take away

The key takeaway is that business and project management skills are critical to the success of any automation initiative. Technical challenges pale in comparison to the issues you will have if you are not aligned with the Business. Don’t start writing code until you have gotten written approval and have a turnkey mechanism to produce the metrics that will satisfy your stakeholders. Remember your project will be judged by the actionable metrics it produced, not by demoing buttons being clicked.

Failure Reason #2: Automation is treated as a time-boxed project and not part of the software development process

Automation is not an 8 week project you can swarm on and then hand off to someone else to ‘maintain’. A common mistake is to take a group of developers to ‘build the automation framework’ and then hand it off to less technical QA to carry forward. Think about that model for your company’s application. Imagine hiring 6 senior consultants to build version 1 in 8 weeks and then handing the entire project off to a junior team to maintain and take forward.

Automation is a software project. It has the same needs for extensibility and maintainability as any other project. As automation is written for legacy and new features, constant work needs to be done to update the framework. New requirements for logging, reporting or handing new UI functionality. As long as you are making changes to your application, you will be updating the automation. Also keep in mind most automation frameworks are tied into other systems (like build, metrics, cloud services) and everything needs to stay in sync as they evolve.

You quickly end up in a situation where junior engineers are in over their heads and either have to stop working on automation until expert resources free up, or they go in and erode the framework with patches and hacks. The end result is conflict which lowers ROI, generates a perception of complexity and difficulty and eventually leads to the failure of the project.

Mitigation

Again, this is an easy pitfall to avoid. Your business plan for automation should include long term resources that stay with automation through its initial lifecycle. It’s still beneficial to bring in experts during key parts of framework creation, but the owner(s) of the automation need to be the lead developers. They will build the intimate knowledge required to grow and refactor the framework as tests are automated.

Additionally, leverage industry standard technologies. Automation is not an area you want to be an early adopter. If your organization is building a web application you will want to pick a framework like selenium instead of something like m-kal/DirtyBoots. A good standard as a manager, you should be able to search LinkedIn for the core technologies your team is proposing and find a number of experienced people in them. No matter how awesome a mid level engineer tell you this new technology is, when he leaves, the next person will insist on rewriting it.

Take away

If you are using standard technologies and industry best practices, you will not need an elite team of devs to build the framework for QA. The complexity for the automation project should remain fairly constant through the life of your company’s application updates, new features, UI uplifts. The original creators of the framework should be the same ones automating the bulk of the tests. Additional, less experienced scripters can be added to increase velocity, but a consistent core group will produce the beast results for the least investment.

Failure Reason #3: Automation is created by a collective without a strong owner who is responsible for standards

Making the automation framework a community project is a very expensive mistake. If your company created a new project initiative with the guideline of “Take 3 months to build a good CRM System in any language that we will use internally” and turned that over to 10 random devs, to work on in their spare time, you would expect issues. Automation has the same limitations. A small dedicated team (with members that expect to carry Automation forward for at least a year or two) has the time to gather requirements, understand the business needs, build up the infrastructure and drive the project forward to success. An ad-hoc group with no accountability, especially one that the main members will not be doing the actual creation of tests, is going to struggle.

Everyone wants to work on the fun POC stage of automation, hacking together technologies to do basic testing and reporting. Most QA has some experience with previous projects and they have their own ideas about what can and can’t work. Without strong leadership, an approved roadmap and strict quality controls you will end up with an ad-hoc project that does a variety of cool things, but you can never seem to tie it together to get the information you need for actionable metrics. The team always has low confidence in the tests or their ability to reproduce results reliably. There is always a reasonable sounding excuse of why. The fun drains away as weeks turn into months and your team finds other things to focus on, while automation stagnates.

Eventually it becomes apparent how little business value was produced for all the effort, how much work remains and no clear owner to hold accountable or plan how to maintain or move forward. The champions for the fun part have shifted their attention to the next cool project. Management will end up assigning people to sacrifice their productivity by manually generating reports, cleaning up scripts and try training others to use it. Eventually everyone agrees the process sucks. Eventually a new idea/technology will surface and the cycle repeats itself.

Another common mistake is assuming that adding more engineers to aid in writing automation will increase ROI. Remember that ROI is measured against the business objective, not lines of code. Unlike application development there are few established patterns when it comes to automation. This means 2 equally skilled automation engineers will write vastly different automated tests for the same features. Remember that adding less experienced engineers requires your experienced engineers to stop building automation and start a training, mentoring and code reviewing program. In order for this program to be successful, every test written needs to be reviewed by 1-2 people to ensure it fits. It will take months until the additional engineers will be autonomous and able to contribute without degrading the framework. Additionally the more complex the application, the more custom systems, features and controls it can contain. Each of these variations will need the more senior engineers to tackle them first. Even with these efforts the business has to accept that new automation engineers will not write the best tests, it can take years to build the skills and apply the concepts correctly. This is a large factor in the constant ‘do-overs’ that automation projects suffer from.

I would assert that the ONLY business value, from automation, comes via the metrics and reports it produces. You could have the best automation in the world, but if it just clicks buttons and never produces an actionable report of its findings, then it has no value. Good automation will be structured in such a way as to produce a comprehensive report, that shows test coverage, is easy to understand and accurate release to release. Imagine having a large group of sales and marketing people, all working separately to generate their own KPIs from their own data. How cohesive would their reports be? Could the business make informed decisions with KPI from different groups at different scopes? The skill to structure and create valuable automation is not the same as being able to read the Selenium Documentation and click a button on a page.

We should always be working towards an approved business objective. Is the business objective to write test cases as fast as possible, even if they can’t be maintained? Or is it to “Automate the Regression to free up QA for other tasks”. Shifting your engineers’ time from running manual regressions, to babysitting automation does not solve anything (and actually reduces the test coverage). In certain cases, slower test case development by a smaller team of experienced engineers is the way to go. As long as you build in redundancy and have their work open for review and feedback you will produce value much faster.

Mitigation

Building automation that can generate reliable and actionable metrics is non-trivial and requires a lot of structure, discipline and previous experience. Automation projects should always be championed by 1-2 engineers experienced with setting up automation projects. They should make a compelling case to the business on what they want to build and what value it will bring. Once the business signs off, they should be given the space to build out the initial POC framework and sample test case(s). Once a working prototype is in place, feedback is solicited and then the project moves forward. The core team should be 2-3 engineers who are equals. Once all the critical areas are automated and the framework is hardened, you can begin training up interested individuals by pairing them with an experienced member of the team.

This initial work should be done by a core team of 2-3 engineers. They will be held accountable for its success or failure. It’s critical to make this group show working tests for all the main/critical areas of the product. It’s these initial tests that expose gaps in the framework. Once a working set of automated tests have been completed and tested from kickoff to report delivery, you can discuss training a small group to start building out test cases and moving automation into other teams.

Take away

When looking at an automation report, you need to be able to understand, at a glance, what was tested and what wasn’t. When you have questions on failed tests, you need to be able to quickly understand what the test did and the results. All tests should have the same scope and voice. Imagine if Feature X only has 5 tests with 100 steps each while Feature Y has 100 tests with 5 steps each, how do you combine those data points to understand the real state of the product. As the group gets larger and larger, it’s harder to maintain a single voice. You will move much faster allowing your core group to solve these problems before introducing less experienced engineer.

Summary

In this post I discussed the three most common reasons automation fails, ways to avoid them, and keep your projects focused on increasing ROI and business value.