A RAMBLE ABOUT METRICS – ZACH BONAKER

MEASURING AGILE

How will you show the progress of agile transformation? How will you measure improvement in teams and people? And how will you avoid the trap of shallow metrics like say:do ratios and velocity?

Whether you’ve worked as a manager, coach, or consultant, you’ve likely experienced conflict and confusion over “agile metrics.” Traditional metrics which emphasize personal productivity drive negative behaviors, encouraging us to stay busy with tasks over working together to achieve goals. Meanwhile, leaders feel dissatisfied with popular “agile metrics,” such as velocity and burn-down charts, when they fail to provide the insights desired.

This conflict between management and the information generated by metrics is further exacerbated when three common human desires for “measuring agile” inevitably occur:

  • The need to prove “the new way of working” achieves more than “the old way of working.” This desire typically reveals itself through challenges like, “how will we know we’re more productive with agile?” or “how can you show me are teams are more efficient?
  • The desire for agile to deliver “more quality.” Often, we hold on to the premise that a new process (e.g., teams will work in sprints) is the key to unlocking quality. The scenario can be especially challenging in the absence of systemic knowledge about quality. “Our CEO is sending strong messages expecting quality to improve. Beyond defect counts and customers complaining, we’ve never had a good idea of what quality is. What do we do?” Sound familiar?
  • The belief that process conformance is a good proxy for success. How we will respond when a senior leader challenges us with, “I want to know the maturity of teams and the adoption of process. How can you show me our agile transformation is successful?

Success Metrics and Improvement metrics

While there are many reasons these scenarios may be challenging–and frustrating–to solve, one possible contributor is the failure to distinguish between “success” and “improvement.”

By definition, “success metrics” are exactly that: the key measurement which reflects on the interaction (and collaboration) within the system to generate a successful outcome. By taking a measure-up approach to avoid focusing on individuals, we can see whether our system of work is delivering the result we need. Because we’re considering the success of the system, we likely only need one, possibly two, success metrics. Further, such metrics would be relatively long-term in persistence.

On the other hand, “improvement metrics” are useful until they no longer are. They exist to help guide us towards reaching a desired state, whether it be behavior, process, or performance. By definition, “improvement metrics” should be relatively short-term in lifespan and may change based on new information gathered. It’s easy to test for an improvement metric: we can state what decision(s) we will make as a result of the metric, plus what conditions would exist for the metric to no longer be useful.

An interesting thought which comes to mind: perhaps in the early stages of using agile (and assuming an absence of prior “success metrics”), our success metric might be the length of time before an improvement metric becomes obsolete.

I believe this structure of categorizing “success” and “improvement” metrics creates an environment where two beneficial things can happen:

  1. We reduce the fear (i.e., gaming) of metrics through transparency of our intent to use metrics. Clear statements of intent, as well as conditions to make metrics “go away,” often create motivation to work with–rather than against–measurement.
  2. The defined difference between “success” and “improvement” metrics can catalyze a helpful mindset change in people.

To illustrate the second point, consider a scenario which might be familiar:

A company shifts from a matrix system of work to teams working in sprints. Once this happens, a manager feels the need to measure team productivity and/or performance. After all, this is a new “process” and the manager wants to make sure it’s effective. To do so, the manager asks teams for a “say:do” ratio: the number of items/points/features/etc predicted versus the total actually delivered.

If not success metrics, it is an improvement metrics

It’s likely the manager considers this measurement a “success metric.” Therefore, we ask her to explain how the metric represents the interaction of all the parts necessary for success. Assuming our goal is wildly successful software products which our customers love, we likely realize this metric ignores essential contributions of product management, customer support, the degree of disruption the team must manage, and other probable factors.

So, if not a success metric, might it be an improvement metric? We explore further:

  • What decisions will we make with this metric?
    • If a “say:do” ratio is considered poor (the team doesn’t meet their forecast), what happens?
    • If the ratio is very good, what then?
    • Could decisions we make with this metric encourage people to “game” the metric or be less transparent?
  • What conditions must exist to make this measurement no longer needed?
    • If a team meets their forecast 100% of the time, we likely no longer need it.
    • Is this condition an indicator we have achieved our goals?

I suspect these conversations might guide the manager to reveal her assumptions about measuring teams aren’t serving her ultimate needs. We then have the opportunity to explore together a more systemic need for measurement, most likely focusing on gathering information to improve, rather than perform.

In my experience with agile and “transformation”, whatever that might be sometimes,  I’ve found results tend to improve when we move away from “metrics to manage behavior” and begin using “metrics to help us understand behavior.” This framework of success and improvement metrics can help us achieve exactly that outcome.

About the Author

Zach Bonaker

Agile Coach

Zach Bonaker is a “benevolent trouble-maker” based in San Diego, California, USA and has more than 10 years of experience assisting organizations with achieving their goals through improved working conditions and team-centric systems of work. With experience guiding Fortune 500 companies to multi-million dollar startups, Zach draws upon agile principles, relationships, and systems thinking to redesign structures into safe, collaborative environments. Zach is an international conference speaker, frequent podcast guest, and contributor to the global agile community. When he isn’t thinking about next-generation agile ideas, Zach can be found enjoying the sunny California weather and connecting with people all over the world.

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments

EXPERIENCE AGILE. START TODAY

0
Would love your thoughts, please comment.x
()
x

Enter E-mail Address

We promise to keep your email address safe. You can check our Privacy Policy.

Patrick Foster

Agile Coach


Patrick Foster

I’m a leader who serves as an Agile Coach in organizations.

I help senior leadership plan long-term strategic decisions while embracing the Agile mindset.

I also work with teams to help them become self organizing on their journey of providing value to the customers.

I see my clients as creative, resourceful, and whole and I’m here to walk alongside them to achieve business outcomes.

Elena Vassilieva

Innovation Leader Coach & Agile Transformation Adviser

Elena Vassilieva is an executive leadership coach who uses virtual reality (VR) and artificial intelligence (AI) to facilitate sustainable and immersive feedback for conscious leadership development. Elena integrates principles of Conversational and Emotional IQ and Systemic Team coaching with an Agile/Lean empirical approach to create a personalized holistic coaching
experience. Drawing on her years of training, mentoring, and coaching executives and teams in Agile transformation, she now serves as a trusted advisor for thought leaders who engage in digital transformation.
Elena is an experienced people development leader known for her ability to creatively align business strategy to people’s practices while keeping a passionate focus on workplace culture and values.
Elena is an ICF Certified Coach, ORSC, and AoEC Certified Systemic Team Coach, Certified Visionary Leadership Coach.
As the Scrum Alliance Certified Coach and Certified Agile Leadership Educator, Elena is teaching, training, and coaching teams and individuals to become great Agile professionals and Agile leaders. She brings together diverse and passionate voices dedicated to developing thriving workplace cultures, creating joy and value with the Agile approach, and embracing the agile mindset in work and life environments.
Elena works with leaders and teams around the globe, helping them engage in digital transformation and mapping out powerful organizational shifts to create a sustainable culture focused on learning and innovation. She helps executives in global companies find alignment around cultural and social diversity and aligns around shared purpose, shared values, and business goals.
Elena blends her diverse experiences as a trusted advisor, professional consultant, coach, researcher, and VR documentary producer.

Alireza Boloorchi

PhD in Computer Science

Alireza Boloorchi is passionate about efficiency for the whole. He believes leadership is about the courage in sacrificing localized efficiency for a greater goal. For the last 12 years, he has been helping several large and small organizations to adopt/adapt more efficient process using Agile/Lean values and principles.

He received his PhD in Computer Science and is Certified Scrum Professional by Scrum Alliance. His experience is rooted in software engineering followed up in roles such as Scrum Master, Agile coach, ENT Agile coach, and leadership in Agile organizations.

He has worked/consulted in several industries such as Finance, Game, Semiconductor, manufacturing , etc. Academia has been an important part of his professional life, and He is researching and teaching Agile in the academia as an Adjunct Professor at Oklahoma State University. And serves as reviewer for several journals such as Journal of Super Computing, Network Systems, and Information Science.

Zach Bonaker

Benevolent Trouble-maker

Zach Bonaker is a “benevolent trouble-maker” based in San Diego, California, USA and has more than 10 years of experience assisting organizations with achieving their goals through improved working conditions and team-centric systems of work. With experience guiding Fortune 500 companies to multi-million dollar startups, Zach draws upon agile principles, relationships, and systems thinking to redesign structures into safe, collaborative environments. Zach is an international conference speaker, frequent podcast guest, and contributor to the global agile community. When he isn’t thinking about next-generation agile ideas, Zach can be found enjoying the sunny California weather and connecting with people all over the world.

Michael de la Maza

PhD, MIT, CEC, Co founder Demingway.com

Michael de la Maza is the founder of Heart Healthy Scrum and a Scrum Alliance Certified Enterprise Coach (CEC). Previously, he was VP of Corporate Strategy at Softricity (acquired by Microsoft in 2006) and co-founder of Inquira (acquired by Oracle in 2011). He is the co-editor of Agile Coaching: Wisdom from Practitioners and co-author of Professional Scrum with TFS 2010 and Why Agile Works: The Values Behind The Results. He holds a PhD in Computer Science from MIT.

Jeff Patton

Chief Troublemaker

Jeff Patton helps companies adopt a way of working that’s focused on building great products, not just building stuff faster. Jeff blends a mixture of Agile thinking, Lean and Lean Startup Thinking, and UX Design and Design Thinking to end up with a holistic product-centric way of working. Jeff is author of the bestselling O’Reilly book User Story Mapping which describes a simple holistic approach to using stories in Agile development without losing sight of the big picture.

Jeff’s a Certified Scrum Trainer, and winner of the Agile Alliance’s 2007 Gordon Pask Award for contributions to Agile Development. You can learn more about Jeff, and find essays and past writing from his columns with StickyMinds.com, Better Software Magazine, and IEEE Software on his website: jpattonassociates.com. 

Amitabh Sinha

Execution Strategy Advisor, CTC

Amitabh (Amit) Sinha is a servant leader entrepreneur, visionary, mentor, trainer and coach. Amit is highly passionate about Agile, it’s principles, values, and the human side. Amit is a people champion and strives to bring out the best in his teams. Amit leverages his expertise in Agile, Scrum, Kanban and people skills to increase team effectiveness and happiness.

Amit has had huge success in transforming teams, departments and organizations. He has led, coached, nurtured and trained hundreds of teams into high performance and happiness! Amit’s teams have often been noted for their high levels of exuberance. Amit helps Scrum Masters, Product Owners, leaders and executives understand the Agile mindset and how to create effective products that delight customers. 

Amit contributes to the community by speaking at various forums and meetups to share his knowledge and wisdom with others. He delivered a popular workshop on happiness and productivity at the global Agile conference August 2019(AA 2019, DC). Amit lives with his wife and three boys in Palo Alto, California.

Rakesh Sadhwani

Business Strategy Advisor

Rakesh Sadhwani is an entrepreneur, businessman, and technical leader who possesses a wealth of industry experience and know-how.  He currently serves as the CEO of an award winning, Bay Area technical staffing and solutions firm, Vertisystem. Rakesh’s expertise includes on-shoring and off-shoring, staff augmentation, business strategy, business development and creating valued partnerships.

What sets Rakesh apart as a leader is his dedication to the relationship first, his genuine commitment to the community, enabling his staff’, and a commitment to doing what is right.  Rakesh is a true partner who sincerely listens and endeavors to deliver value above and beyond your expectations. Rakesh lives with his wife and two daughters in Fremont, California.