How We Actually do it: Accounting for Bugs in an Agile World

Mike,

You are right, bugs fixes do get counted in the team's sprint capacity. You set aside a certain amount of capacity (story points per sprint) to fix defects from old code. You estimate bug fixes in story points, and include them in the sprint. You fix the bugs and track them on the sprint burn-down. This determines the teams sprint capacity.

The distinction is that you aren't making progress if you are fixing bugs. Time spent on bug fixes is time spent not moving new features into production. A Product Owner can't determine the progress we are making towards a product release and therefore make accurate decisions without understanding the rate we can move working (bug-free) code into production. So bugs fix time doesn't count as project velocity and its not additional work on the release burn down. You get better velocity by producing less buggy code.

In the case where you get done early, you should have some extra backlog groomed and fit that into the sprint. If you are producing more production code (not fixing stuff that is broken) you get credit for it and your velocity will exceed plan. Do this often enough and you can increase velocity for planning purposes. So you know how much capacity you have, and you set some aside some of it for defects. But you don't count your bug fix time towards team velocity for product planning purposes.

>>I know many coaches will argue that bugs should be tracked seperately and that bugs should be dealt with in the sprint that they're found. However in practice this is not always practical

I have just one question. Do you have Definition of Done for user stories? If you have it, how it happens that you do not have "Zero Bugs" rule in your DoD?

>> Many times you find bugs after you have deployed to production and the customer starts using it.

Of course it happens and even more often than desired ;)
There is HUGE difference between bug found during story development and bug found by customer. I did not get the distinction in the article. Bug found during story development is MUCH easier to fix, since you have focus on the story, you know what you've done 2-3 days ago and everything is fresh in mind. If bug found next month, it will take time to switch to it, include into the new build, deliver, etc.

>> Additionally, some bugs you may decide to push out (they may be minor).

To be honest, we've tried to push out minors bugs. As a results during 1 year we accumulated about 100 bugs (with Normal and Small severity) that live in the database in Open state. Can the application run without these bugs? Yes, it can. Is it good? No, it is not. This practice does not work for us and currently we have zero bugs rule in DoD. BTW, it'll take time to get rid of all these bugs accumulated during 2 years.

>>Secondly, during the sprint, when you find bugs they need to get added just like tasks do and you need to account for how long it's going to take to fix. This will cause the burndown total hours to increase. on that day that you find them.

No objections here if you track burn down in hours. But there is a simpler approach. Normally you have quite the same development/bug fixing ratio for each iteration. So you may just run 1-2 iterations and define the time you need to fix all discovered bugs.

>>I personally am not fond of running specific bug fix sprints but this is an approach that many companies follow and more often than not, companies have to do it this way.

C'mon! That's plain wrong practice! It is just a bug fixing phase in waterfall. It contradicts to agile development nature. How it sounds for example "well, waterfall is bad, but companies have to do it this way".

I think we should not try to advocate bad practices in any case, but just say that they are bad and should not be followed. No excuses.

Best,
Michael Dubakov
http://twitter.com/mdubakov

I like that you are talking about this subject and I'm not sure there are any "correct" answers and more ideas to implement to find what works best for you. Here are some quick thoughts I had while reading the article.

Zero Defect Mentality
The current approach you are using has the assumption that defects will continue to be found at a pace to have a sufficient future impact. I think once you take the approach that defects should really be a rare occurrence and that defect free software development is possible it will change how you manage this problem. I like the comment that talks about "done is done" problems. If you are seeing a high rate of defects when things are going into production it is a symptom that your current process is broken in some way.

Defects Should Be Estimated
I guess when you get to the point where you feel its viable to be defect free then you come to the conclusion that original story points are in fact the measure of future defect estimation. That is you do not estimate defects because it means you didn't actually correctly complete stories that had estimates marked as completed originally. If you are forced to consume those defects it gives you a more realistic picture of what your velocity really is...

Before you completely freak on this concept.. if a team starts to write sloppy software in order to increase velocity.. ie: I increase my velocity from 12 to 20, but am writing really bad software... Then bugs come back three iterations later... I am able to continue my 20 point velocity without it raising concerns because I adjust for all the defects showing up. If instead I was forced to consume them. My velocity would drop significantly. The product owner would freak and we would have to discuss why we are seeing so many defects.

Consider Slack
One thing I have seen is teams add in a small amount of slack to their iteration. So while they might be able to do a 22 velocity they commit to say a 20 instead. This allows them to deal with defects, tackle code debt (refactor), do research or deal with issues that were not for seen.

I am not a huge fan of the slack method and I hate the estimate/include method. I guess I just believe in defect free enough to inflict pain that forces us to deal with it. I loved the post and that someone is talking about it.

BTW, quote from Lean Software Development by Mary & Tom Poppendieck :)

"One of the fastest ways to kill motivation is what is called in the US Army a zero defects mentality. A zero defects mentality is an atmosphere that tolerates absolutely no mistakes; perfection is required down to the smallest detail. The army considers a zero defects mentality to be a serious leadership problem, because it kills the initiative necessary for success on a battlefield"

Hi Jack (et al),

Firstly, I'd like to say this is an interesting article that raises several interesting points. Some of these points are generally agreed upon in the wider scrum community and some are not. Rather than try and address all the relevant points here, I'd like to address just two of them below:

1. My personal recommendation for *customer* report bugs (ie. existing or legacy bugs that are in production) is that they should become a story in the product backlog and dealt with just like any other story (estimate in Story Points, prioritized by the PO, etc). The purpose of this is to make them visible to the Product Owner and force him/her to prioritize customer reported defects in relation to new functionality. This is consistent with Ken (Schwabers) Squirrel Burger story.

Known bugs found this sprint (for software in development) need be fixed before a Story can be considered "Done." And, should only be accepted by the PO once it's "Done."

2. In my experience, capacity planning is best done using "yesterdays weather" (http://c2.com/cgi/wiki?YesterdaysWeather) using story points. I would strongly recommend against using hours for capacity planning, mainly because there is no correlation between developer hours and software in production (http://jeffsutherland.com/scrum/2007/10/why-time-sheets-are-lame.html).

>My company is often working on many simultaneous projects and having to juggle resources.

I may be reading this incorrectly, so please forgive me if I've taken it the wrong way ... If by "resources" you mean "people", then you're not doing scrum. Scrum teams are by definition dedicated to a team.

Best regards,
Kane Mar (CST, CSC, CSP ... etc)
http://KaneMar.com | http://www.linkedin.com/in/kanemar
http://www.twitter.com/Kane_Mar | http://www.twitter.com/AgileCarnival

>>If you saw one of Jeff Sutherlands more recent video's .. he talks about rolling scrums .. part of his scaling scrum message .. is that developers may be working on multiple simultaneous sprints. I know it's hard to wrap ones head around this but it is possible. I wouldn't advocate this though.

It seems we are falling back to waterfall and multi-tasking...