Software Development

A Software Metrics Primer, Part 2

By |

The first article in this two-part series suggested some specific metrics that can help any software-developing organization better understand how it operates and the results it gets. This article will present some suggestions for creating a healthy measurement culture in the organization and give you several practical tips for metrics success.

Creating a Measurement Culture

A software practitioner’s first reaction to a new metrics program often is fear. People are afraid the data will be used against them, that it will take too much time to collect and analyze the data, or that the team will fixate on getting the numbers “right” rather than on building good software. Creating a healthy software measurement culture and overcoming such resistance will take diligent, congruent steering by managers who are committed to measurement and sensitive to these concerns.

To help you overcome the fear, educate our team about the metrics program. Tell them why measurement is important and how you intend to use the data. Make it clear that you will never use metrics data either to punish or reward individuals—and then make sure you don’t. A competent software manager does not need metrics data from individuals to distinguish the effective team contributors from those who are struggling.

It’s also important to respect the privacy of the data. It’s harder to misuse data if managers don’t know who the data came from. Classify each base measure you collect into one of these three privacy levels:

• Individual: Only the individual who collected the data about his own work knows it is his data, although it may be pooled with data from other individuals to provide an overall project profile.

• Project Team: Data is private to the members of the project team, although it may be pooled with data from other projects to provide an overall organizational profile.

• Organization: Data may be shared throughout the organization.

As an example, if you’re collecting work effort distribution data, the number of hours each individual spends working on every development or maintenance phase activity in a week is private to that individual. The total distribution of hours from all team members is private to the project team, and the distribution across all projects is public to everyone in the organization. View and present the data items that are private to individuals only in the aggregate or as averages over the group.

Make It a Habit

Software measurement need not consume a great deal of time. Commercial tools are available for measuring code size in many programming languages. Activities such as daily time tracking of project work constitute a habit each developer gets into, not a burden. Commercial problem tracking tools facilitate counting defects and tracking their status, but this requires the discipline to report all identified defects and to manage them with the tool. Develop simple tracking forms, scripts, and reporting tools to reduce the overhead of collecting and reporting the data. Use spreadsheets and charts to track and report on the accumulated data at regular intervals.

Tips for Metrics Success

Despite the challenges, many software organizations routinely measure aspects of their work. If you wish to join this club, keep the following tips in mind.

Start Small. Because developing your measurement culture and infrastructure will take time, consider using the Goal-Question-Metric method described in the first article in this series to select a basic set of initial metrics. Once your team becomes used to the idea of measurement and you’ve established some momentum, you can introduce new metrics that will provide the additional information you need to manage your projects and organization effectively.

A risk with any metrics activity is dysfunctional measurement, in which participants alter their behavior to optimize something that is being measured instead of focusing on the real organizational goal. As an illustration, if you’re measuring productivity but not quality, expect some developers to change their programming style to visually expand the volume of code they produce, or to code quickly without regard for defects. I can write code very fast if it doesn’t have to run correctly. The balanced set of measurements helps prevent dysfunctional behavior by monitoring the team’s performance in several complementary dimensions that collectively lead to project success.

Explain Why. Be prepared to explain to a skeptical team why you wish to measure the items you choose. They have the right to understand your motivations and why you think the data will be valuable. Be sure to use the data that is collected, rather than letting it rot in the dark recesses of a write-only database. Encourage team members to use the data themselves and to identify other measures that will help them improve their work. When my small software group at Kodak established a work effort metrics program long ago, we found it to be more helpful than we expected. One team member expressed it well: “It was interesting to compare how I really spent my time with how I thought I spent my time and how I was supposed to spend my time.”

Share the Data. Your team will be more motivated to participate in the measurement program if you tell them how you’ve used the data. Share summaries and trends with the team at regular intervals and get them to help you understand what the data is saying. Let team members know whenever you’ve been able to use their data to answer a question, make a prediction, or assist your management efforts.

Define Metrics and Procedures. It’s more difficult and time-consuming to precisely define the base and derived measures than you might think. However, if you don’t pin these definitions down, participants are likely to interpret and apply them in different ways. As a result, you might end up combining apples and oranges, which yields fruit salad instead of meaningful insights. Define what you mean by a “line of code” or a “user story”, spell out which activities go into the various work effort categories, and agree on what a “defect” is. Write clear, succinct procedures for collecting and reporting the measures you select.

Understand Trends. Trends in the data over time are more significant than individual data points. Some trends may be subject to multiple interpretations. Did the number of defects found during system testing decrease because the latest round of testing was ineffective, or did fewer defects slip past development into the testing process? Make sure you understand what the data is telling you, but don’t rationalize your observations away.

Measurement alone will not improve your software performance. Measurement provides information that will help you focus and evaluate your software process improvement efforts. Link your measurement program to your organizational goals and process improvement program. Use the process improvement initiative to choose improvements, use the metrics program to track progress toward your improvement goals, and use your recognition program to reward desired behaviors.

Also Read A Software Metrics Primer, Part 1

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

Latest posts by Karl Wiegers (see all)