Danilo Sato

Inspired by a post on the Lean Blog, Pat reminds us that you can’t measure everything effectively. Having written my Master’s thesis on metrics for Agile projects, I’ve learned and read about this in a lot of different places. One approach that is very known to Empirical Software Engineering researchers is the Goal-Question-Metric approach, first published by Vitor Basili et al. in the 90’s.

The GQM model suggests a hierarchical view of three levels to define which metrics to use:

• Conecptual level (goal): the motivation for measurement. Measuring things without a purpose and a thorough understanding of the problem will lead to meaningless metrics. This level imposes the hardest questions: what’s the purpose? what’s the object of measurement (your product, process, people)? what’s the motivation? who is interested in this goal? what are the quality attributes?
• Operational level (question): at this level, a set of questions are defined to try and correlate the object to the quality attributes we are interested in. These questions should help in understanding and assessing the current situation, but also in identifying ways to determine whether the goal is achieved.
• Quantitative level (metric): only then a set of metrics is associated to the questions, to try and find a quantitative way to measure and answer it. These can be objective (like code coverage), or subjetive (individual’s ranking of current code quality). Finding these metrics is not easy either.

It’s easy to try to cut corners and get into the things that are easy to measure first, specially when you can collect lots of quantitative data to work with these days. However, if you you don’t stop to think about the goals and motivations for measurement, it’s easy to forget the systemic complexity that surrounds us and look only for the easy-to-track numbers.

Lean management and problem solving is known for taking a very thorough and detailed approach in the understanding phase. To many people this is a paradigm-shift approach to management. Don’t let the numbers fool you, use them to your advantage.