A Meaningful Dashboard for Agile Teams

Metrics are important for Agile teams. It is wiser to focus on a handful of simple balanced metrics rather than hoard of metrics that does not give much value to the team or organization. In fact, it creates more noise. 

A good team creates the metrics to understand how it is doing, what it needs to improve and makes the metrics available to the entire organization in the spirit of visibility and transparency. Stakeholders keep an eye on the metrics to see how the team is performing on its commitments. The leaders should review the metrics from time to time. Any question on an odd looking metric should be followed-up with a sense of concern. Their focus should be on helping the team, empowering the team and supporting it to move towards excellence rather than judging, discriminating or criticizing. The Servant Leadership role, such as Scrum Master or an Agile Champion, is needed to bring in effectiveness by understanding the metrics and using the data to help the team grow in effectiveness. 

An ideal dashboard should kindle curiosity, emotions and questions. If any stakeholder questions the relevancy of data, the Scrum Master and the team should review the feedback objectively and update the dashboard as appropriate.

While there are many attractive metrics available, care should be taken to choose a few meaningful ones, especially when a new team has just begun its journey.

Here are some considerations to make while deciding your set of metrics.

Value - Are we giving high value to our end users and customers? How can we show or display this on our dashboard?

Predictability - What is the ability to deliver on commitments?

Productivity - Consistently delivering same or more amount of work in the timeboxed period (iteration).

Quality - How free from defects are the products we build?

Sustainable Pace -  The teams should continue to work at a sustainable pace, indefinitely. 

Happiness (Growth) - Are the people and teams learning, growing and happy?

 Let us now dive a little deeper into what metrics can represent the above themes.

Value can only be measured after the product has been released in the market. Did we meet our desired outcomes? If yes, then we got high value from our product. If no, what are we doing about meeting our desired outcomes in the upcoming sprints? Does our current/future work reflect the feedback that will help us reach or move towards our desired outcomes?

Predictability should be measured as a trend. We should measure the last Sprint Predictability and also the Average Predictability from the last 3, 5 and 10 Sprints. This will show the trend. If the trend is consistent or slowly becoming better, it’s a good thing. If it’s going down, or it’s random all the time, that should be of concern to the team and for the leaders to see what the team needs. Maybe the team needs coaching and understanding on how to deliver to the commitment.  

It could also be that the product owner and the business side need coaching, as they could be putting too much pressure on the team during the sprint to take on additional or non-committed work after the sprint starts, thereby disrupting the rhythm. The Scrum Master may need coaching if he/she is commanding the team to self-organize, when the team is still in the forming/storming stage and may benefit more from direction than just delegation (at the current time). 

Productivity is measured in terms of Velocity. We recommend that as in case of predictability,  the team should display its last sprint velocity and also the average velocity for past 3,5 and 10 sprints. Velocity should also be reported as two numbers. Most people are familiar with velocity as the total number of points delivered in a sprint. However, working with many teams over the years, I have come to realize that while the total points delivered per sprint may be close to the committed points, the actual points delivered were not the same as the committed points or the points committed at the beginning of the sprint. This shows that either the planning is not robust or there are impediments in the system or the culture that disrupts the planned work more often than not. As a result, even high performing teams may be exchanging their committed scope to deliver a different set of work than they planned for. True velocity numbers, can uncover a lot of hidden problems in teams and how they work. It could also mean that the backlog items are not being groomed properly. Working with unhealthy and incomplete backlog items, teams often find they are unable to proceed with a story they started, forcing them to exchange the incomplete story with another one in the backlog. Velocity numbers are never to be compared between teams, although a team can benefit from comparing its own velocity between sprints. 

Quality: As a team becomes effective  and displays Agile Values and Principles in its behaviors (example: collaborating, storming, swarming, cross-functionally diverse, carrying a musketeer’s attitude, etc.), they take pride in their work and that reflects  not only in them meeting their commitments but delivering work with a high quality. What this means is that the production defects should continue to decline, and should ideally drop close to zero with time. Metrics such as “Defects found in the past 3, 6 and 10 sprints” should reflect exactly that. “Meantime to resolve defects”, especially P1 and P2, should be minimized, which also shows proactiveness of an Agile team in resolving past defects and creating less technical debt. 

Code-complexity is another important metric that can be measured. There are tools that help measure cyclomatic complexity, which actually tells the risk inherent in any build, determining how complex the code is. A team should also display a Sprint Report for the last completed sprint where some of these numbers are displayed in a table format; example total points committed, points added after sprint started, points removed from the sprint, total points completed, actual committed points completed, etc. - give a hoard of information on how the team is doing and uncovers hidden problems that are not visible by just a single number.

It is a good practice for a team to also  display their top three or four retrospective  improvement items that they identified in the last sprint (retrospective) showing progress for each item (example: completed, in-progress, disowned or debunked). This tells the stakeholders and Agile champions for the team how the team owns its continuous improvement and how dedicated it is towards improving its behaviors and processes. 

All the metrics mentioned above reflects on the past team performance. It tells you how much the team completed, how much percentage of work it completes every sprint, quality of work, etc., but the one and the only metric that is forward-looking is the Team Happiness. As Jeff Sutherland explains, this can reflect on how the team will do in the future. The diagram below shows a lower team happiness can impact other metrics negatively (in the near future).

How do you measure team happiness? Happiness is one of the things you cannot measure so easily. You cannot quantify happiness in a very direct manner. It has to be measured by taking inputs from the team, and by observing their actions, behaviors, body-language, communication, etc. This is what the Servant Leader Scrum Master or Agile Champions in the bigger team have to look out for. They have to connect with members one-on-one and with the team as a whole, continuously working towards increasing and sustaining team happiness. While working with my Scrum teams for years, I would ask every team member at the end of retrospectives to write a number (on a flip-chart) from 1-10 indicating how happy they are working as a team. Whenever this number would go below seven, I would take personal care to connect and understand what could be bothering the team or the team members and would work towards bringing the happiness back up organically. I always secured a budget from the leadership to celebrate with the team. This would often result in a small celebration either at the work or outside and would include things like food, drinks, movies, team building outings, etc. 

Here is a snapshot of the dashboard.

Conclusion:

You can improve only what you can measure. Meaningful and useful metrics uncover problems that reduce the team effectiveness, and are therefore, needed for continuous improvement.

I recommend starting with the dashboard and metrics explained in this article. The dashboard above also includes the sprint burndown chart and a few other things that I found useful. These metrics are not all readily available from any one given tool. While working with industry standard tools like Jira, Rally and others, we were able to extract a few of the metrics, while others had to be created using third-party tools such as easyBI. Metrics such as retrospective items and Team Happiness were input manually by the Scrum Master.

Amitabh (Amit) Sinha is a servant leader entrepreneur, visionary, mentor, trainer and coach. Amit is highly passionate about Agile, its principles, values, and the human side. Amit is a people champion and strives to bring out the best in his teams. Amit leverages his expertise in Agile, Scrum, Kanban and people skills to increase team effectiveness and happiness. See more

0 0 votes
Article Rating
Subscribe
Notify of
guest

2 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
John

Amit,
Great article. I really liked the idea of team happiness and retrospective items. Securing a budget to celebrate is another good idea but my boss says completing committed work in a sprint is compensated by paycheck. So, why should we have a budget for celebration ? Any thoughts ?

EXPERIENCE AGILE. START TODAY

2
0
Would love your thoughts, please comment.x
()
x

Enter E-mail Address

We promise to keep your email address safe. You can check our Privacy Policy.

Patrick Foster

Agile Coach


Patrick Foster

I’m a leader who serves as an Agile Coach in organizations.

I help senior leadership plan long-term strategic decisions while embracing the Agile mindset.

I also work with teams to help them become self organizing on their journey of providing value to the customers.

I see my clients as creative, resourceful, and whole and I’m here to walk alongside them to achieve business outcomes.

Elena Vassilieva

Innovation Leader Coach & Agile Transformation Adviser

Elena Vassilieva is an executive leadership coach who uses virtual reality (VR) and artificial intelligence (AI) to facilitate sustainable and immersive feedback for conscious leadership development. Elena integrates principles of Conversational and Emotional IQ and Systemic Team coaching with an Agile/Lean empirical approach to create a personalized holistic coaching
experience. Drawing on her years of training, mentoring, and coaching executives and teams in Agile transformation, she now serves as a trusted advisor for thought leaders who engage in digital transformation.
Elena is an experienced people development leader known for her ability to creatively align business strategy to people’s practices while keeping a passionate focus on workplace culture and values.
Elena is an ICF Certified Coach, ORSC, and AoEC Certified Systemic Team Coach, Certified Visionary Leadership Coach.
As the Scrum Alliance Certified Coach and Certified Agile Leadership Educator, Elena is teaching, training, and coaching teams and individuals to become great Agile professionals and Agile leaders. She brings together diverse and passionate voices dedicated to developing thriving workplace cultures, creating joy and value with the Agile approach, and embracing the agile mindset in work and life environments.
Elena works with leaders and teams around the globe, helping them engage in digital transformation and mapping out powerful organizational shifts to create a sustainable culture focused on learning and innovation. She helps executives in global companies find alignment around cultural and social diversity and aligns around shared purpose, shared values, and business goals.
Elena blends her diverse experiences as a trusted advisor, professional consultant, coach, researcher, and VR documentary producer.

Alireza Boloorchi

PhD in Computer Science

Alireza Boloorchi is passionate about efficiency for the whole. He believes leadership is about the courage in sacrificing localized efficiency for a greater goal. For the last 12 years, he has been helping several large and small organizations to adopt/adapt more efficient process using Agile/Lean values and principles.

He received his PhD in Computer Science and is Certified Scrum Professional by Scrum Alliance. His experience is rooted in software engineering followed up in roles such as Scrum Master, Agile coach, ENT Agile coach, and leadership in Agile organizations.

He has worked/consulted in several industries such as Finance, Game, Semiconductor, manufacturing , etc. Academia has been an important part of his professional life, and He is researching and teaching Agile in the academia as an Adjunct Professor at Oklahoma State University. And serves as reviewer for several journals such as Journal of Super Computing, Network Systems, and Information Science.

Zach Bonaker

Benevolent Trouble-maker

Zach Bonaker is a “benevolent trouble-maker” based in San Diego, California, USA and has more than 10 years of experience assisting organizations with achieving their goals through improved working conditions and team-centric systems of work. With experience guiding Fortune 500 companies to multi-million dollar startups, Zach draws upon agile principles, relationships, and systems thinking to redesign structures into safe, collaborative environments. Zach is an international conference speaker, frequent podcast guest, and contributor to the global agile community. When he isn’t thinking about next-generation agile ideas, Zach can be found enjoying the sunny California weather and connecting with people all over the world.

Michael de la Maza

PhD, MIT, CEC, Co founder Demingway.com

Michael de la Maza is the founder of Heart Healthy Scrum and a Scrum Alliance Certified Enterprise Coach (CEC). Previously, he was VP of Corporate Strategy at Softricity (acquired by Microsoft in 2006) and co-founder of Inquira (acquired by Oracle in 2011). He is the co-editor of Agile Coaching: Wisdom from Practitioners and co-author of Professional Scrum with TFS 2010 and Why Agile Works: The Values Behind The Results. He holds a PhD in Computer Science from MIT.

Jeff Patton

Chief Troublemaker

Jeff Patton helps companies adopt a way of working that’s focused on building great products, not just building stuff faster. Jeff blends a mixture of Agile thinking, Lean and Lean Startup Thinking, and UX Design and Design Thinking to end up with a holistic product-centric way of working. Jeff is author of the bestselling O’Reilly book User Story Mapping which describes a simple holistic approach to using stories in Agile development without losing sight of the big picture.

Jeff’s a Certified Scrum Trainer, and winner of the Agile Alliance’s 2007 Gordon Pask Award for contributions to Agile Development. You can learn more about Jeff, and find essays and past writing from his columns with StickyMinds.com, Better Software Magazine, and IEEE Software on his website: jpattonassociates.com. 

Amitabh Sinha

Execution Strategy Advisor, CTC

Amitabh (Amit) Sinha is a servant leader entrepreneur, visionary, mentor, trainer and coach. Amit is highly passionate about Agile, it’s principles, values, and the human side. Amit is a people champion and strives to bring out the best in his teams. Amit leverages his expertise in Agile, Scrum, Kanban and people skills to increase team effectiveness and happiness.

Amit has had huge success in transforming teams, departments and organizations. He has led, coached, nurtured and trained hundreds of teams into high performance and happiness! Amit’s teams have often been noted for their high levels of exuberance. Amit helps Scrum Masters, Product Owners, leaders and executives understand the Agile mindset and how to create effective products that delight customers. 

Amit contributes to the community by speaking at various forums and meetups to share his knowledge and wisdom with others. He delivered a popular workshop on happiness and productivity at the global Agile conference August 2019(AA 2019, DC). Amit lives with his wife and three boys in Palo Alto, California.

Rakesh Sadhwani

Business Strategy Advisor

Rakesh Sadhwani is an entrepreneur, businessman, and technical leader who possesses a wealth of industry experience and know-how.  He currently serves as the CEO of an award winning, Bay Area technical staffing and solutions firm, Vertisystem. Rakesh’s expertise includes on-shoring and off-shoring, staff augmentation, business strategy, business development and creating valued partnerships.

What sets Rakesh apart as a leader is his dedication to the relationship first, his genuine commitment to the community, enabling his staff’, and a commitment to doing what is right.  Rakesh is a true partner who sincerely listens and endeavors to deliver value above and beyond your expectations. Rakesh lives with his wife and two daughters in Fremont, California.