Should fixing bugs count toward velocity?
Key Considerations:
- Why do we measure velocity in the first place?
- To help with forecasting an expected project completion date
- To indicate how many stories we need to prepare for the next time box
- To provide information on how many stories we can reasonably commit to in subsequent time boxes
- Avoid creating a phoney sense of progress. Velocity, being a count of running, tested features, was always about creating a concrete progress indicator as opposed to phoney, misleading progress based on documents and the like.
- Highlight rather than hide failure demand. We want to highlight work that is value recovery rather than value adding so that we can target it for process improvement.
My Typical Suggestion:
Imagine that we're running down a field toward a goal at the end. If our time box is one minute then our velocity is the distance we've travelled toward the goal divided by one minute. The important part here is "toward the goal".
Let's imagine that we run 10 metres but then someone pops up and pushes us back 5 metres. However, before the minute is up we recover those 5 metres. We've expended 15 metres of effort but we're only at the 10 metre mark.
What's our velocity then?
It's 10 metres / minute NOT 15 metres / minute because we are measuring progress toward a goal, not effort expended. Velocity is a vector, not a scalar.
So, should fixing bugs count toward velocity? No, we are measuring progress toward a goal, not effort expended.
In practice, including bug-fixing effort in velocity might actually have a negligible impact on our planning. However, including bug-fixing in velocity will also have a tendency to hide it, which is undesirable from an improvement perspective.
It's an interesting question. Personally, I would count bug fixes towards velocity - but, in the classic tradition of double-entry accounting, I'd also want to take that velocity from the earlier iterations where the bugs were created, and apply it as negative overhead to this one.
ReplyDeleteThe conversation would go something like this:
"Last iteration, we completed 15 points of work. So we scheduled 15 points for this iteration.
Unfortunately, we uncovered defects in that work. This meant we only delivered 10 points of work this iteration. The defect-related work was estimated, in iteration, at about 5 points worth.
Next iteration, we will plan for 10 points of work, and reserve 5 points of our expected velocity of 15 for defect fixing - and prevention"
By tracking it this way, you expose the cost of defect fixing, both in time spent and in opportunity cost (Hey, we could have done 5 more points if we didn't have those bugs!). By allocating the same amount of time to fix bugs next iteration, and using it for preventative work if bugs aren't found, you'll help address the cause of the bugs in the first place, and pick up that velocity later.
I've thought of doing something like what you've described where we are explicit about the portion of velocity contributing to value-adding work vs the portion contributing to value-recovery work.
ReplyDeleteI've never actually done this in practice though.
In your example I would subtly change the approach to say our velocity was 10 points per iteration and we also had 5 points of defect fixing. I am slightly wary that we assume that the actual velocity with no defects would be 15. It could be that we need to slow down to prevent defects and yet end up with faster actual velocity. So instead of a speed of 15, and a velocity of 10 (with 5 points of defects), we might go with a speed of 13 and a velocity of 13 (with 0 points of defects).
I count bug fixes when they are "pointed" fixes - we also assume a certain amount of emergent bug fixes that are done in the normal course of business). In addition to Robert's points it lets decisions be made about bug fixes that are good to do but not required. I can see value in separating out that work for the point you make about progress.
ReplyDeleteWhat do you do with "infrastructure" work - things not asked for by the client but required to deal with say the traffic increases on the site? I also count that in velocity.
Nice post. I like the running towards the goal metaphor.
ReplyDeleteOne aspect that's a bit more in the gray area here is technical tasks. Many recommend not estimating tech tasks. They can sometimes be considered as moving towards the goal (e.g. setting up a UAT environment) and sometimes not (e.g. refactor that class that doesn't adhere to our MVC architecture).
As for bugs, I usually recommend avoiding estimation. Oftentimes the bulk of the work is in diagnosis - reproducing the problem, ruling out possible causes, examining the code, etc. Once the diagnosis is done, you can estimate the effort to fix the problem but you've failed to account for the diagnosis effort, which can be significant.
For this reason, I usually produce a separate burn-up of bugs (by count, not by estimate) so the team can see how the bug count grows and is remediated. As more bugs are addressed, the velocity will go down. The bug discovery and remediation rates should be fodder for retrospectives where conversations can be had about root cause and actions to take to address (e.g. Jason's slowing down approach). As always, the team is likely the best source of ideas on how to address the problem.
curiouscat, Adrian: I would typically include infrastructure / technical tasks as part of existing feature-based stories or have separate stories which means counted in velocity.
ReplyDeleteWhat is "the goal?" How do you know it is the right or best goal? I think the "goal" is usually multi-dimensional, unless your goal is simply "we want X story points worth of non-bug-fixing features in our release." How do you know that you won't produce features that the customers don't want and won't pay you for. Suppose your velocity is progress in the wrong direction. For many cases, the goal isn't simply software with features.
ReplyDeleteMaybe the goal is new features. Maybe the goal is to produce one feature with many customer exciters. Maybe the goal is to fix bugs in a legacy product. The practice of not counting bugs towards velocity may work for your team, but there are many contexts in which this does not make sense. Suggesting a blanket, generalized practice like this could be extremely dangerous for many contexts.
Suppose you have many defects in production from a legacy product. You have major, long-term customers threatening to leave unless you harden the software and fix the bugs. I would posit that counting bug story points toward velocity in this context would be extremely important. I think it would be irresponsible to not count that work as "progress toward your goal."
As an aside question, if bug fixes are important enough to estimate and commit to completing in an iteration, then why would that not count as progress towards a goal? Is relatively bug-free software not a goal?
Greg: I'd agree that it is possible that what we believe is the goal during development turns out to be incorrect after we've released to customers. I would generally deal with that in by 1. adjusting how we design and user test changes and by 2. incorporating feedback from releasing. The first part will be reflected by velocity, the second part will be reflected by changes in the target line.
ReplyDeleteIn the case where we are talking about pre-existing bugs, not bugs produced by the current development team, I'd probably treat them like new stories and incorporate them in velocity. It would be useful to distinguish them to see how effort is allocated.
Failure demand is important enough to address. It only counts as progress toward our goal if we ignore counting the failure as negative progress first.
Interesting perspective.
ReplyDeleteAccording to the Scrum Alliance:
http://scrumalliance.org/articles/39-glossary-of-scrum-terms#1110
Velocity is: "how much product backlog effort a team can handle in one sprint"
So, if a defect is a prioritized back log item that goes into the Sprint... then it counts.
If you're talking about bugs that come up during the development of a story/back log item (PBI), then it wouldn't count - as the PBI factors everything needed to get to "done" - which... includes all bugs being cleared.