Setting up the metrics that helps Scrum Team can indeed be a very difficult job and, in this article, I will explore some things to keep in mind while you are setting up your Metrics
- Metrics Are Not About Keeping Things Green – One common anti pattern that I see in many Scrum Teams is where the focus of metrics reporting is to showcase that team is always “within acceptable boundaries” This is usually seen when stringent metrics are defined in agile contracts and are tied with financial penalties. These types of metrics encourage the Scrum Teams to represent the data in way that makes the metrics always seem “green”. In such a case the focus cannot be on what we need to do in order to improve. Where possible, I try and coach the teams not to have these service level agreements in their Agile Contracts – I encourage the teams to look at metrics as data points that will help them to improve rather than a tickmark needed for contractual compliance.
- Focus on End-to-End Metrics – It is usually a better idea to have metrics which measure the value end to end across the value chain rather than measuring value at interim points. To give an example, metrics measure end to end time required for a feature rather than tracking time for a specific task like coding or testing are more useful. Another example can be, number of zero defects features delivered is a better Metrics than number of defects identified during testing phase. When defined in this manners Metrics encourage Scrum Team to work as a unit rather than individual sub functions. It becomes a challenge to define such metrics in a multi-vendor scenario.
- Measure Only What You Want to Improve – Metrics are not meant to be “frozen forever” – Often, I see exhaustive metric structures that take up a lot of time and energy for the Scrum Team to create and maintain the data. These metrics frameworks often are collecting data without having a plan of how the data will be used to help the Scrum Team to be more effective and efficient. I have seen teams that are lost in this huge data that they maintain. These teams struggle to understand the data and make any useful inferenve from that. In my experience, it is often easier to have a smaller number of metrics that the Scrum Team has to track. This way the team can actually use the data being captured. Some of the most effective organization and Scrum I have had good success with Scrum Teams that decide their improvement goals for the upcoming period of 6 months to an year and then determine the metrics that will help them track their journey towards these these goals. These teams periodically review their goals and adjust the metrics accordingly. This also aligns with the Scrum Value of Focus
- Avoid metrics that Compare one team or one person against another – Agile believes into collaboration. A scrum Team needs to wok in a Self managed collaborative manner to deliver value frequently. If the metrics measure individual(or sub team) performance, it may sometimes lead to people working against each other to protect their own Metrics level. It is a better idea to measure value delivered as the whole team. The same principal can be applied to creating a competition between the teams- for large products requiring many teams it can hamper the collaboration amongst Scrum Teams. It works better if the teams are measured against their own last performance and encourage to have a continuous improvement in their way of working rather than comparing against other teams.
- Trends Need To Be Monitored – NOT individual numbers – Sometimes a particular metric may show a bad value. This can happen because of multiple transient reasons and it may not be worth it to really worry about an isolated value or reading in a metric. It is much more useful to monitor the trends and analyze if the Scrum Team is going in the right direction.