My First Agile Project Part 5: Our Top 5 Agile Mistakes

September 29, 2008 by mattgrommes

Picture courtesy of DWQ Online@flickr

In the previous parts of this series (1, 2, 3, 4), I went into a lot of the initial issues of how we ran our project and some of the things we did wrong. For Part 5, I'm going to focus on the 5 big mistakes we made in the project before I move onto another phase of the series.

For some background, the project was to integrate an off-the-shelf (but highly customizable) billing system to replace an aging custom Oracle forms app. This was our company's first foray into Scrum and Agile as well as the first Agile experience for most of the development team. After we sailed past our second management-imposed deadline I started thinking about how we could correct our process for this project and the next ones we do, which led me to write it all down in this series. Keep reading for the Top 5 mistakes we made (in no particular order). Where possible I also give some detail about how we've worked on correcting our process. I hope this list helps you avoid these mistakes and make your own all-new ones. :)

1) Not tracking backlog
This was by far our biggest mistake I think. As I mentioned in Part 1, we failed to keep track of how much work we were adding to our backlog as we went along. We all assumed that since we were being Agile, we would add work as we fleshed out requirements and learned new capabilities. The problem came with not tracking how much work we were actually adding and comparing it to how much we were doing in each sprint. Without doing this, another of our mistakes - Scope Control - became much more of a problem than it might have. Not reassessing the backlog periodically meant we didn't know how scope creep was really affecting the project, which meant a downward spiral of allowing the scope to creep, not knowing the effect of the new scope, allowing scope to creep more, etc., etc.

Since we realized a few months ago we should have been doing this, we've been keeping much better track of what gets added to the backlog and making sure we're being as ruthless as we can about moving features to Post Go-Live and only working on important bugs. As a result we have a much better view of the finish line of the project.

2) Too much planning upfront, not enough later
In Part 2 I talked about our Inception Phase where we had our initial meetings to get a rough estimate of the integrations we would be doing and business requirements from other departments who would be using or relying on the billing system we were building. The mistake we made here was in the amount of requirements gathering we did and how we did it. We took a "snorkel, not dive" approach where we didn't get too many details, which is a good approach. But we also made requirements documents complete with flow charts of integration program flow. These documents took much too long to write and didn't end being used by the developers at all. Once we started sprinting and doing the work, we re-learned, or un-learned, pretty much everything in the documents but it all made more sense since we had the context of our code to work in.

Obviously not getting deep technical requirements up front means you need to get them later and this is part 2 of this mistake. We should have made it our practice to start getting requirements for the next sprint so we could start working without waiting. As it happened, our Product Owner/Subject Matter Expert was busy enough that we spent a lot of time waiting for requirements before we could start.

3) Boring business people with technical demos
I was tempted to make this mistake "Didn't get management involved properly" but I think the early mistake was our demos. As I said in Part 4, we demoed everything we did that sprint, including infrastructure and behind-the-scenes integrations. This is good for the team to get to show off but it's boring and turns off the business people and management. Once you've turned people off the process, it's hard to get them back into it when you need their input. They're still going to give you input, it's just going to be later than you expect and usually after you think you're done with the thing they should have seen 3 sprints ago.

Making the demos boring for the business folks also meant we failed to properly engage our upper management. We ignored this problem for too long and once they started getting more engaged, it was after the deadline had been blown and they weren't aware. Not a good position to be in. Having more business focused demos wouldn't have guaranteed anything but combining better demos with an insistence that the business / management people attend would have helped immensely.

4) Not estimating product backlog items
Part 1 of this series talked about how we were Doing 80% of Scrum. What we missed was a chunk of related, but separate processes around estimating and planning the backlog. This mistake is tied to a couple of the others because of this but we committed them all separately so I'm listing them separately. Our product backlog was basically just a list of work we had to do. Our Product Owner prioritized her most important items before the sprint planning meeting, then we moved the items we were going to do over and estimated them then. We didn't estimate the backlog items ahead of time so she could know how hard we thought things were or so we could know how much more work was ahead of us. We had rough ideas that Integration X would take longer than Configuration Y but without real estimates, we couldn't start harder things earlier in the process or push things that were hard and low priority off until later, both of which are vital for planning.

Since we're in bug-fixing mode right now, we're struggling with fixing this mistake more than with any of the others. I got a bunch of Planning Poker cards from Mike Cohn at Agile2008 and we've been trying to use those but it's hard when the issues you're estimating are bug fixes that only the person who worked on it originally knows about. If you have any suggestions on this, I'd love to hear about it in the comments.

5) Scope control
Not controlling the scope of the project, combined with our lack of estimates and not monitoring the growth of the backlog, constitute by far the largest cause of problems with our project. If we had been estimating our backlog items it would have helped us track the amount of work we had left, which would have helped us know when and what to remove from the backlog and keep to the next release. Since we didn't do any of those things, the backlog grew too much, which pushed back the set of testing sprints we had scheduled, which pushed back the work that testing revealed, which put us in the position we're in today, with a late project, lower morale, and lost credibility with management.

One factor I'm struggling to figure out is that the nature of our project seems to limit our ability to control scope. We're doing an integration of a new billing system to replace an old system. This means there's a set of existing functionality we have to duplicate in the new system that can't be pushed to the next release. Also, being a billing system you have to get things right the first time. You can't send out a partial invoice, for example, just because doing the whole thing will take too long. But I'm sure everybody thinks there are exceptional things about their particular project so I don't want to fall into the trap of saying we can't learn anything about how to control scope because our project is the exception. There's always something that can be pushed back and if you're doing the right things with estimation and monitoring your backlog it makes finding those things much, much easier.

We've been in bug-fixing / testing mode for the past few months and have tried much harder to keep things out of the backlog if they didn't 100% need to be there. There's still things going in, but now we make sure each thing is absolutely needed for go-live and everything else goes on the list for after.

Mistakes != Fail

We've made mistakes on our project, for sure. We're not done yet so I'm sure we'll make more. But we're learning, which is the important thing for our next projects. As a team we're all committed to Scrum and being Agile so we're going to make the effort to improve our processes where we think it'll help. The more technical-oriented aspects of Scrum are incredibly appealing to us as developers. Keeping a backlog, committing to work, keeping interruptions to a minimum, sprints; all of this we've done mostly correctly and we love how it's worked over the past year and a half.

I've spent the last 5 parts of this series talking about the basics of how we ran our project and the lessons we've learned while we chugged along sprinting. Next time, I'm going to start talking about what happened once we missed our first deadline. We changed our process, halfway-unknowingly moved away from Scrum, and suffered for it until recently when we did a reset of our process and got mostly back on course. Stay tuned next week and as always, I'd love to hear any comments you have on this series. Thanks for reading.

My First Agile Project Series
Part 1: Doing 80%
Part 2: Inception & Planning
Part 3: Viral Videos and Bad Jokes in Scrum Demos
Part 4: How to lose credibility and jeopardize your project with lack of management buy-in
Part 5: Our Top 5 Agile Mistakes
Part 6: The First End Of Our Project

Permalink