Useless Agile Metrics

Ideally, a metric is a leading indicator for a pattern change, allowing your Scrum team to analyze the cause in time and take countermeasures. However, what if you picked the wrong metrics? What if these useless agile metrics lead you in the wrong direction while providing you with the illusion that you know where your team is heading?

Learn more about useless agile metrics, from individual velocity to estimation accuracy to utilization rates.

Useless Agile Metrics in Detail

Several metrics can be misleading or counterproductive for agile teams, indicating that the organization might not fully grasp agile principles and instead adheres to traditional industrial approaches and Taylorism. Some of these metrics include:

• Individual Performance Metrics or Velocity: Measuring individual productivity or comparing team members based on hours worked, lines of code, or tasks completed can harm collaboration and teamwork, and it goes against the agile principle of focusing on the team’s collective success.
• Total Number of Story Points Completed: Focusing solely on the number of story points completed without considering the delivered value or complexity can lead to a “quantity over quality” mentality and encourage teams to prioritize less valuable tasks.
• Velocity Comparisons Between Teams: Comparing the velocity of different teams is not valid, as each team has its unique context, skills, and challenges. Comparing velocities may lead to unhealthy competition and encourage teams to inflate estimates, also known as cooking the agile books.
• Estimation Accuracy: Agile practices value delivering working software over comprehensive documentation. Therefore, spending excessive time and effort to make “estimations accurate” is less valuable than delivering working product increments and gathering feedback.
• Utilization Rates: Focusing on how busy team members are rather than the value they deliver can lead to overemphasis on task completion and hinder collaboration among team members, the ability to respond to changes, and adapt to evolving priorities.
• Fixed Scope and Deadline Adherence: Insisting on strict adherence to predefined scope and deadlines will discourage teams from adapting to changing requirements, a core principle of any agile practice.
• Project Success by Planned vs. Actual Metrics: Traditional project management focuses on executing according to plan, but Agile values responding to change over following a plan. Thus, the success of a project cannot be measured merely by comparing planned vs. actual timelines or budgets.
• Number of Defects: Measuring the number of defects as a primary quality metric can incentivize teams to focus on short-term fixes rather than addressing the root causes of issues and investing in long-term improvements. Also, it may encourage agile teams to play “bug-free” instead of addressing potentially highly valuable yet technically challenging customers’ problems.
• Percentage of Time Spent on Planned vs. Unplanned Work: While tracking and managing unplanned work is essential, an overemphasis on this metric can discourage teams from responding to new opportunities or addressing emerging risks.

These metrics, while they may provide some information, do not necessarily indicate the health or effectiveness of an Agile team. They might even lead to behaviors that contradict Agile principles. To foster a genuinely agile mindset, organizations should focus on value-driven, outcome-based metrics that encourage collaboration, continuous improvement, and adaptability. Examples of more valuable agile metrics include lead time, cycle time, customer satisfaction, and business value created.

Origins of Useless Agile Metrics

Several factors contribute to why organizations and managers may insist on applying less useful agile metrics:

• Legacy Mindsets: Managers from traditional hierarchical structures may struggle to let go of legacy metrics. These metrics are often ingrained in the organization’s processes and are used to evaluate performance and make decisions.
• Perceived Control: Metrics such as “utilization” or “work hours” can give managers a false sense of control and predictability. They may believe they can control productivity and outcomes by managing these quantifiable elements.
• Lack of Understanding of Agile Principles: Managers may not fully grasp the principles of agility. They may not understand that agile emphasizes customer value, team collaboration, adaptability, and continuous improvement, over traditional parameters like strict schedules and scope.
• Fear of Change: Changing to new metrics can be uncomfortable. It requires the organization and its managers to reevaluate their performance standards, which can be challenging.
• Misalignment of Incentives: Sometimes, managers are incentivized based on these legacy metrics. When their rewards or promotions are tied to these measures, they naturally lean towards using them.
• External Pressures: Stakeholders, clients, or investors used to traditional metrics may demand them, putting pressure on the managers to comply.

Consequently, organizations must educate their managers and stakeholders about agile principles and demonstrate how new metrics align better with business objectives, value creation, and embracing business agility in general than the described useless agile metrics.

How To Overcome Metrics Anti-Patterns in Agile Organizations

Addressing the issue of less useful metrics involves strategic planning, communication, education, and patience. Here are some practical steps a Scrum Master can take:

• Build Trust: Foster a strong relationship with management based on trust. Understand their concerns and what they are trying to achieve with these metrics. Provide reassurances that agile metrics can deliver the same, if not better, insights into performance and outcomes.
• Educate: Regularly educate the management about agile principles, its values, and how they translate into practice. Explain why certain traditional metrics are not suitable for agile teams and how they can lead to wrong decisions and behaviors. Highlight the benefits of using agile-friendly metrics. For example, organize workshops or seminars with an industry expert or agile coach to better communicate the value of suitable metrics in agile environments.
• Share Success Stories: If other teams or organizations have successfully transitioned to better metrics, share these stories. Showing that it can be done and the benefits that resulted can help convince skeptical managers.
• Introduce Meaningful Metrics Gradually: Start introducing more meaningful metrics to the mix, explaining what each one means and how it aligns with agile principles. Over time, as they start to see the value in these metrics, they may feel more comfortable letting go of the less useful ones.
• Promote Transparency: Be open about the purpose, benefits, and limitations of each metric used. Gather and present data that supports your arguments about the ineffectiveness of certain metrics and the benefits of those you propose. A transparent approach promotes understanding and acceptance.
• Demonstrate the Impact of Wrong Metrics: Provide real-life examples or case studies demonstrating how inappropriate metrics have led to counterproductive behaviors and adverse outcomes. It’s often more convincing when they can see the negative effects for themselves.
• Involve Teams in Discussions: Bring the teams into the discussions around metrics. They can provide valuable insights into how certain metrics affect their work and morale. Their direct feedback can help management understand the impact of their decisions.
• Patient Persistence: Change takes time, and old habits die hard. Keep communicating, educating, and demonstrating patience. Persistence can pay off in the long run.

Aside from the practical steps already outlined, there are a few other points you might consider when dealing with the issue of useless agile metrics:

• Understanding Resistance: Often, the insistence on using traditional metrics is rooted in a resistance to change and fear of the unknown. It’s essential to understand this aspect and address these concerns empathetically.
• Culture Change: Moving from traditional to agile metrics isn’t just about changing the numbers we track but a shift in culture. It might take longer than you expect and require much persistence.
• Continuous Improvement: Agile is all about continuous improvement. Regularly review the metrics you use, seek feedback from all parties involved, and be open to making changes as necessary. Remember, what works for one team might not work for another, so customization is critical.
• Lead by Example: Show management the value of agile metrics by implementing them in your work and demonstrating their effectiveness. Leading by example is a powerful way to influence change.

Finally, remember that communication is vital. Regular, open, and honest discussions about the metrics, why they matter, and how they contribute to team success and organizational success will go a long way toward gaining acceptance of more agile-friendly metrics in general and avoiding useless agile metrics altogether.

Conclusion

Abandoning useless agile metrics and adopting more agile-friendly ones requires shifting the organizational culture and addressing resistance to change with understanding and empathy. It’s a process of continuous improvement that requires patience, persistence, and customization.

The Scrum Master plays a crucial role in leading this transformation by advocating for relevant metrics and demonstrating their value in action. Open, honest, and regular communication about the importance of these metrics and their contribution to success is essential.

Ultimately, transitioning to agile metrics requires more than a change in the numbers we track; it demands a shift in mindset, fostering an environment where progress and value take precedence over merely being busy. In other words, we must abandon our Taylorist heritage in that respect.

How have you managed to shift your organization’s culture to replace old performance metrics with useful agile metrics? Please share your thoughts with us in the comments.