Who’s driving the bus?

imageHere at DTS, we’re very focused on consulting-type software development. As such, we have very direct access to our end users and customers. Our work is “clearly” defined and prioritized by our customers and we receive direct customer feedback every two weeks. We do not have a dispersed customer base, it’s usually a single organization. However, last week I had lunch with a friend who does more “shrink-wrap” development. His customers and end-users never define or prioritize their needs. In fact, unless it’s by pure happenstance, the developers never meet or know their customers. The functionality and feature set for the software is defined by an internal customer proxy who has his “finger on the pulse of the customer”.

Now a disclaimer: I’ve never worked on a shrink-wrap team, I’ve always been on consulting development teams. Given that little disclaimer, I don’t understand how a customer proxy can really define what an entire unseen, unknown customer base really needs. I know that some organizations do prototype or “focus-group” type testing to gauge what their customers want. But my friend’s team develops on a gut instinct of what their customer base needs. I’m not sure this ever can truly deliver quality and value to a wider audience. I think it leads to Microsoft Word-type deals, with tons of functionality that only a few people actually use. Don’t get me wrong, I think a lot of good things come out of gut instincts. I think there is a lot of value to anticipating user needs. It’s part of innovation.

The other point my friend made was that in the shrink-wrap world of DVD-based delivery systems, organizations are hard pressed to keep up with the rapidly changing demands of large customer bases. Their release schedules and update maintenance schedules must be horrendous. They can’t respond rapidly to changing or evolving customer needs. The best they can do is provide patches and updates via the web. New functionality is released on the next release…on DVD…every 9 to 12 months.

I do see a solution to this problem and I think the folks at Rally really embrace the ever evolving needs of their dispersed customer base. It’s how they gather and prioritize their customer’s needs that impresses me the most. Rally has a community website where users can enter feature requests. The other users in the community can vote on the usefulness of the feature requested and provide additional feedback and comments on the feature request. Based on these entries and their associated popularity, Rally is able to understand their user needs and prioritize them. Many of the features requested by their user community show up in very frequent releases of their web-based solution. No DVD’s or anything. Just a quick release and instant functionality to their customers. Speaking from personal experience, I’ve seen functionality we’ve asked for implemented within a single 7-week release from when we requested the feature.

To me, this is as close as you can get to your customers if you’re doing “shrink wrap” software development. For those of you out there doing shrink-wrap development in an agile manner, how do you engage your customer base to deliver high value, quality software? I’m very curious how customer proxies work in agile organizations? How do you define and prioritize your customer needs if you never get to interact with your customers? When I think of these types of questions, it makes me very happy to be on a consulting development team. Sometimes I think I take the close relationships we have with our customers for granted. After this conversation with my friend, I think I’ll value the collaborative nature of our relationship with our customers a whole lot more.

Book Review: Agile Management for Software Engineering

AgilemanagementI have finished reading Agile Management for Software Engineering, a book by David J. Anderson, and I am very impressed. Granted, the book is already four years old (I’m a little behind with my reading) but it’s still relevant. It is the first book I’ve read that tries to back up the agile approach with solid theory. In this book David uses the Theory of Constraints to explain why lean/agile project life cycles lead to success, and a higher return on investment (ROI), more often than traditional life cycles. He also presents an interesting in-depth analysis of Scrum vs. XP vs. FDD.

However, since I am Dutch, you can usually expect me to have found something to complain about (or disagree with). And, being the considerate person that I am, I would never try to disappoint you. So, here are my three gripes:

  1. David doesn’t hide the fact that he has a preference for Feature-Driven Development (FDD). After all, he has been practicing it for many years. And I have no trouble with that, except that I felt he was favoring FDD a little too often with some of his analysis. Whatever the subject, and whatever his reasoning, FDD always seemed to come out on top. It seemed as if an Italian was trying to prove that Rome is the greatest city in the world. (Which it isn’t. It’s London.) Though I could not bother myself to find any specific flaws in David’s logic, I did find it a bit suspicious.
  2. Another thing that annoyed me was his view on the complexity of software projects. At one time David admits that software projects are nonlinear (and therefore complex) by nature, while on other pages David tries to convince his readers that his linear estimation technique (involving feature lists) has produced accurate results for years. I think this is a contradiction. You cannot have an accurate linear estimation technique for a nonlinear phenomenon. It’s like estimating the noise Italians will make on a bus by counting the number of Italians. (Everybody knows the correlation is exponential.)
  3. However, the worst thing about the book is that it wasn’t funny. It does not contain any jokes. You might find this strange, but I always need a bit of lightness to help me through the tough parts in a book. Unfortunately, there was not one little bit of silliness to help me crawl through 300 of pages of dry theory. David made me feel I was back on the university; the only difference being that David’s formulas weren’t hand-scribbled in illegible writing and photocopied with a twenty year old copier. I think, after having reached the last page of the book, I let out such a big sigh of relief that I must have left my neighbors wondering.

Nevertheless, if you’re looking for a book that does more than simply saying "Agile is great, and traditional methods stink!" then I suggest you give it a try. The book is solid and important. Go buy it!

Video tutorial: Managing your Scrum Product Backlog with the help of a simple Excel sheet

‘;
} else {
print ‘
‘;
}
?>

In this tutorial I demonstrate how you can manage your Scrum Product Backlog in a very simple Excel sheet based on a popular free template. Watch this screencast if you want to get started with Scrum. You will see how to get started, how to detail, inject and remove requirements, how to build burndown and velocity charts.

You can download the somewhat lower quality version of the tutorial from Google Video.
. For a larger version click on the post title’;
}
?>

See also

Not Repeatable Results but Repeatable Success

BrainI don’t believe in repeatable results. I only believe in repeatable success.

Repeatable Results vs. Repeatable Processes
In his recent Agile Newsletter, using the results of the latest Dr. Dobb’s survey, Scott Ambler showed his readers that most developers, project managers and IT managers prefer repeatable results over repeatable processes. Not surprisingly, according to his survey results, people in the agile community lean more towards repeatable results than those working in traditional organizations. Most respondents of Scott’s survey also reported that their personal preference for repeatable results is bigger than that of their organization. Everything considered, it seems that there is a global shift towards a growing importance of repeatable results vs. repeatable processes.

But what are repeatable results?

Is the human brain a repeatable result of biological evolution? Is Google a repeatable result of our global economic system? Is the Pokémon hype a repeatable result of our international social system? The answer is ‘no’ in every case. Is Safari for Windows a repeatable result? I certainly hope not! (And it appears I’m not the only one.)

No Repeatable Results
Software projects are complex adaptive systems, just like species, organizations and societies. Richard Dawkins and Stephen Jay Gould, both famous evolutionary biologists, have both argued that if you were to "replay" evolution, the brain (and human beings for that matter) would probably not appear again. Our existence on this planet was made possible thanks to many happy (or unhappy) coincidences, like a meteor killing all the dinosaurs, and butterflies flapping their wings at the correct times. Repeating evolution would possibly yield some other intelligent form of life, but it wouldn’t be human. It probably wouldn’t even look human! (And there definately would not be a Scott Ambler nor a Jurgen Appelo to bother you about repeatability of results.)

Likewise, I don’t believe that replaying the birth and growth of the Internet would give us another Google. Sure, there would be search engines, but the playing field would be very different. And I’m afraid that my position as CIO of our organization is also not a repeatable result. Five years ago I met our (former) CEO at my cousin’s birthday party. I don’t even like family birthday parties! So there’s little chance of that happening again…

Projects Are Sensitive to Initial Conditions
Complex systems are very sensitive to initial conditions. It means that, no matter how often you replay the same project, even when all the processes are the same, the outcome will be different every time. If you were to replay any successful software project, you would never get the same product. Just a tiny variation in the constraints will give you very different results. Replacing just one person in a self-organizing team would yield very different ideas and solutions. Even having the coffee machine replaced with another brand can have a huge impact on your results. (I know it happened with me.)

Repeatable Success
Nevertheless, even though the results of your software projects are not repeatable, project success is repeatable! No matter how many times you replay evolution, the result will always be an ecosystem of successful species. No matter how often we restart the big bang of the Internet, there will be successful companies making money from advertising. Principles like self-organization and emergence, which are essential for evolution, capitalism and agile software development, mean that success is virtually guaranteed. Agile processes and principles will give you successful software projects. But nobody knows what the results will look like.

Agile software development is not about repeatable results. It is about repeatable success.

Of course, I am sure that this is what most people mean when they talk about repeatable results. But it’s my opinion that this slightly adjusted view may help improve our communications. In fact, the Butterfly Effect tells us that the effect of this little contribution could be very well be tremendous. Let’s wait and see…!

Investing in customers

Braun WK210 Aqua Express
Many Finnish companies have a tradition of periodically selling the old stuff to its employees on the internal auctions. If you are lucky, you can get a decent computer, printer or fax for a couple of euros. Several years ago the company I was working for (CIM Wireless) has been bought by a bigger one. We were going to move to the new company premises and therefore had a particularly big auction where they were selling pretty much everything older, than a year. At that time I needed a new kettle and got one right from the company kitchen for as little as five or ten euros. I didn’t put much thinking in, I just needed a new kettle – it didn’t really matter which one.

When quality matters

It happened, however, that such a commodity as an electric kettle could be of so high quality that it makes you notice it. We didn’t examine the device or measure its performance, but we couldn’t miss such features as boiling water fast or ability to turn off automatically, when was empty. It might sound amazing, but we were actually surprised by the fact that it just worked very well. Plus it looked nice. We even recommended this model to a couple of friends – something you rarely do about kettles.

When I managed to accidentally break it I went to the store looking for the very same model. I was able to find it in the second shop, but I was ready to visit couple of stores more, before considering another model (by the same brand – that was not a question at all).

Pay-off

Investing in the product or brand quality is often like the story above. You sacrifice some immediate benefits, because of investing extra funds or time in building quality in a product, might get no exceptional return for a reasonable amount of time and then “suddenly” figure out that many loyal customers want to purchase an upgrade from your company.

The bottom line: If you company is here for more, than a single release of a single product, care about the long-term users. Otherwise Braun will do it for you.

Your company

What does your company do with your kettles? Do your quality assurance related activities consider issues that are likely to happen in two years after a sale?

Fooling Myself with False Velocity

Writing2I’m an idiot, I have fooled myself. I’m usually quite good at fooling others, but this time I have fooled myself. And I did it with what I might call a false velocity.

Velocity Defined
Velocity is the rate at which you complete your work, in a certain period of time. This might refer to the number of stories completed, the number of issues processed, the number of exams passed, the number of kilometers driven, the number of hamburgers baked and served, or (for some guy I know) the number of girls dated and dumped.

The Velocity of Writing a Book
I might have mentioned earlier that I’m trying to write a book. It’s a book about managing software development, and the things we can learn from complexity science. Last year I did a lot of research, a tremendous effort that will continue to the end of this year. And I started writing my first pages of text in January this year. I am planning to write around 350 pages in 1.5 years, which equals to an average of 5 pages per week. After 15 weeks I have completed 75 pages of text, which is… 5 pages per week! It’s exactly the velocity I need to finish the book as scheduled. Right?

Wrong!

Last week I came to realize that I still haven’t finished the latest book that I had picked up two months ago. In fact, I still want to read another 15+ books that I consider essential for my project. It now appears that my writing is on schedule simply because I have been neglecting my reading! I have deluded myself. My velocity is all wrong!

The Trap of Measurements
I sometimes see people make a similar mistake in their software projects. It is the trap that people can fall into when measuring progress. People tend to optimize their work according to that what is being measured. And they tend to neglect what is not being measured. In my case, I measured the writing of pages of text. So that’s what I did. I wrote pages of text. My measurements did not include finishing my research. So I neglected it. I fooled myself into thinking that I was on schedule. I’m an idiot.

Measure Everything!
In any software project you have to take care to measure everything that is essential in delivering a succesful product. If you measure only functionalities, you might neglect your work on quality stuff. And if you measure only story points, you might forget about time that must be spent on research, education or communication. I don’t care if progress in your project requires issues processed, exams passed, kilometers driven, hamburgers baked, or girls dated and dumped. Measure everything that counts! Don’t fool yourself as I fooled myself.

Note: I will not be bothering the readers of AgileSoftwareDevelopment.com with unrelated material. If you want to see more details of how things are turning out with my project, you can follow my efforts, hope and despair at Noop.nl.

10 Key Principles of Agile Software Development

10 Key Principles of Agile Software Development, and how Agile Development fundamentally differs from a more traditional Waterfall approach to software development, are as follows:

1. Active user involvement is imperative
2. The team must be empowered to make decisions
3. Requirements evolve but the timescale is fixed
4. Capture requirements at a high level; lightweight & visual
5. Develop small, incremental releases and iterate
6. Focus on frequent delivery of products
7. Complete each feature before moving on to the next
8. Apply the 80/20 rule
9. Testing is integrated throughout the project lifecycle – test early and often
10. A collaborative & cooperative approach between all stakeholders is essential

To read more about these principles, see here:
http://kw-agiledevelopment.blogspot.com/2007/02/10-things-you-need-to-know-about-agile.html

Kelly Waters
www.allaboutagile.com

XP Practice: Pair programming

Pair programming is one of the most known Extreme Programming practices. In essence it means creating the code in pairs trying to get multiple perspectives on the code created. One of the participants, called driver, types the code and thinks about the low level details. Another one, called navigator, thinks about what the typed code is for. The participants periodically switch roles. The definition of “low level” depends on the skills of the concrete pair and their experience in the subject area. For example, when using test-driven development the driver might be focused on the writing the production code, while the navigator could be thinking about how the next unit test should look like in order to change the system architecture in the desired direction.

In my own practice I found that besides leading to better quality because of continuous code review, pair programming is a very efficient tool for rapid competence transfer and for synchronizing the people views on both the tasks and the solutions. Even if the team is not regularly doing par programming, pairing for some time in the beginning of the project or after the major decisions, helps aligning the team members understanding. Also being paired with the more experienced developer, typing under his governance and seeing which low level tricks he uses is a very efficient way to learn and align the team standards for coding.

Further links

This page is a part of the Extreme Programming overview

My Dad was a ScrumMaster

PanAm In light of the recent news about American Airline’s maintenance problems, I sat down and talked with my Dad about aircraft maintenance. You see, my Dad was a maintenance Crew Chief for Pan American Airlines for nearly 30 years and I wanted to understand how things got so bad for American. My Dad explained the inner workings of Pan Am’s maintenance program to me. The program dated back to late 1960’s and early 1970’s and was remarkably agile-sounding. Here’s the program in a nutshell.

Aircraft are regularly called in for routine servicing. The servicing was usually scheduled for about 3 days. A QA specialist (or several QA specialists) examined the aircraft and used a set of cards to write up work orders for each item that required servicing. The QA specialist then sorted the cards by theme (interior, engines, avionics, etc.), estimated how many items could be completed in 3 days based on historical data, and slotted them on a task board. The crew chief for each “theme” area would pick up the cards for his team. The crew chief and his team would triage the work items and prioritize them. Then, the team members would select the work items they would work on and provided the crew chief with time estimates to complete each of the tasks they selected. This allowed the mechanics doing the work to actually estimate the duration of their tasks. The team would then commit to completing the tasks within the 3 day service period. Each day, they’d check in with each other to make sure things were on track. During the service period, the crew chief worked on some maintenance tasks, but he also made sure that everyone on his team had what they needed to get their job done. At the completion of the service period, the QA specialist would come back and inspect each maintenance item and either accept or reject them (and FAA inspectors checked things as well). Team members were also encouraged to think about how they did their job and suggested new or better ways to do things (in fact, they received monetary incentives for coming up with innovative ideas).

Now, I don’t know about you, but this sounds very much like Scrum to me. If you think about it, we have a highly complex piece of equipment with multiple integrated systems. You have a mission critical maintenance program. We have high priority items that need to be completed…you know done done…to keep the aircraft safe and airworthy (and other that are lower piority like a reading lamp that isn’t working). You can’t mess up here. So how do you do it effectively and efficiently: Scrum. Here’s how I see it:

We have a time boxed iteration (the 3-day service period). We have a product owner (the QA specialist), a ScrumMaster (the Crew Chief) and a team (the mechanics). We have a prioritized backlog (the task board and task cards) and team members selecting their tasks, providing task estimates, and committing to a team goal of having the aircraft ready to roll in 3 days. We have a daily stand up (checking in to make sure they’re on track). We have a servant leader ensuring his team has what it needs to meet their goal and we have a product owner review at the end of an iteration. And most importantly, we have retrospection and continuous improvement. Seems like Scrum to me.

So, maybe Scrum is just a repackaging of things we already knew that worked well. It’s just being applied to software development now instead of aircraft. Toyota picked up on much of this and Taichi Ohno implemented lean manufacturing there in the early 90’s with a lot of similar tactics. Maybe we shouldn’t be too proud of ourselves for discovering something new…we really just rediscovered something old that worked and there’s nothing wrong with that. So, let’s file this one under “Nothing New Under the Sun”. Or better yet, let’s file it under “My Dad Was a ScrumMaster”, he just didn’t know it.

Why Managing Small Projects is Harder

DedicatedserverSome people think that managing small projects is easier than managing big projects. They are right, unless the economies of scale require you to manage a number of those small projects at the same time. In that case, managing small projects is a lot harder than managing a big project!

Dedicated servers vs. shared servers

Many web sites are too small to run on their own dedicated servers. I know, I’ve built a lot of those sites. They usually run on shared servers (together with other web applications for different customers). As many system administrators know, management of a shared server is a lot more work than management of a dedicated server. On a shared server, the applications are fighting for the same resources (memory, drive space, database connections). And we must try to make sure that, when something goes wrong in one application, it doesn’t bring down all the others too. Unfortunately, this does happen occasionally. One time it might be because of a Denial-of-Service attack against one particular site, another time it’s simply because I screwed up again with one of my infamous infinite error loops. Customers often complain when problems with another site causes a failure on theirs. We then kindly suggest that they open up their purse and get themselves a dedicated server. Or, as a bus driver might say, "If you can afford yourself a taxi, sir, then why take a bus?"

Dedicated teams vs. shared teams

I see the same problems with software development teams. I have worked in a number of organizations where projects where too small to form dedicated teams. These organizations usually work with shared teams. The resources in a shared team have to be shared among different projects. The customers simply don’t pay enough to give them the full attention of one entire team. Management of such a shared team is a lot more work than management of a dedicated team. The projects are fighting for the same resources, and you must try to make sure that, when something goes wrong in one project, it doesn’t take down the other ones. But again, this does happen every now and then. Customers complain about it, for sure. We then kindly suggest that they buy themselves a dedicated team, which they never do. They prefer complaining over paying twice the fee. Just like ordinary bus passengers.

Agile methods assume a taxi, not a bus

There are many shared teams in the world, and not just in software development. Many hair dressers, car mechanics, accountants and lawyers work in shared teams. Such a team doesn’t serve just one customer at a time. (Except for teams of lawyers with a customer the size of Enron.) Each team member usually serves a different customer, and resource planning can be cumbersome, because of the many cross-customer interdependencies. Unfortunately, every agile method assumes that customers have bought themselves the privilege of working with a dedicated team. It’s as if they assume that people always take a taxi, and nobody takes a bus. It’s a pity, because managing a bus is a lot harder than managing a taxi.

Can agile principles help a shared team?

Yes, I’ve seen that it is possible, but it takes a lot more effort. First of all, it takes one very strong project manager (or Scrum Master) to negotiate alotted time with many different customers (or Product Owners). Second, you have to translate all best practices for dedicated teams to the shared sitation. (Stand-up meetings are slightly different. Backlogs are treated differently. Iterations are very different. Pair-programming works in some cases, but not all. Etc.) Third, you have to educate anyone who thinks that you’re "only doing simple projects". Because, when you can manage a number of small agile projects simultaneously, then you are able to manage anything!

Any bus driver can drive a taxi. But few taxi drivers can drive a bus…