The 7 Software Development Wastes – Lean series Part 4 – Transportation

Introduction

Previous posts on the first 3 wastes can be found here: In-Process Inventory/Partially done work , Over Production/Extra Features and Extra Processing

It’s both interesting for me on the one hand, yet puzzling for me as to what makes a blog popular or not. The first article I wrote on this topic had close to 5000 reads as compared with only 1000 for the last blog post. I figured that based on the response to the first post, folks would be real keen to understand the rest of the 7 wastes in software development. Never the less, I am committed to continuing with the series. Hopefully there are still some of you out there who can benefit from it all.

Transportation – Hand-offs

Transportation in manufacturing corresponds to hand-offs in software development. Anytime you hand a deliverable off to a different party, there is some loss in the transfer of knowledge.

There are many such examples of hand-offs in software development:

1. Developer hands off to another developer. In this scenario if the first developer never documented the code properly it’s going to take significant addition effort to figure things out. Worse, the second developer may make assumptions and, as a result, introduce bugs in the system.

2. Developers hand off code to testers to test. Many organizations still don’t engage QA early enough. Bear in mind that on Agile projects, QA is involved as early as the requirements phase (not that it’s a phase). If the QA has no idea what the developer did or the problems he faced or the assumptions he made, then the QA is really just shooting in the dark. It’s important that the developer at least includes the QA early on, documents the feature accordingly (no long dissertations), so that an effective transition is made.

3. Handing the code over to deployment teams. Many times I find separate deployment teams struggling to figure out how to get applications deployed. Configuration settings, compile instructions etc if not properly communicated can cause significant delays.

4. Handing-off to customers. If the client is not trained properly, or the software functionality is not documented properly, there will be more support calls for example.

There are many ways in which to reduce transportation wastes in software development:

1. Ensure there is open communication between all parties. Broadband communication (as defined in Agile software development practices) is imperative.

2. Ensure proper, up-to-date and effective documentation is in place where appropriate. Identify these needs up front and ensure there are tasks for this on the backlog.

3. Include all functional areas in the development process.

Be mindful of and identify all these transition points in your organization. This will help minimize costs due to hand-offs in your organization

Next week I will cover the 5th waste in the series – Motion

Fixed price part 1, what’s so bad?

In his excellent article 10 Contracts for your next Agile Software Project, Peter Stevens described 10 different contracts that are often used in software development projects. At the end of his article he singles out one contract-type:

“Fixed price projects and agile development are considered not to mix. I know from experience that this is not strictly true. I am also convinced that other forms are better.”

I completely agree with him on this but since fixed price contracts are so common in software development it seems like a good idea to look at them a bit more carefully. In this article I want to look the problems with fixed price and why traditional software engineering methodologies are unable to solve these problems. In my next article I want to continue on a more positive note and show how agile practices will help you make the most of fixed price.

What’s so bad?
The most obvious downside is of course lack of flexibility. Fixing project scope in the contract often makes it non negotiable during the project often preventing change altogether. In my experience stakeholders will change their minds during any non-trivial project and fixing scope at the start of a project will limit the quality of the end-product because it will not fit the client’s wishes one-hundred percent.

Not only scope is fixed at the start of a project but price is also fixed, hence the name. Making the supplier to commit to price estimates even before the project has started forcing critical decisions at the worst time possible. The later you make a decision the more information you have to base that decision on. Agile tells us to make them at “the last responsible moment”. Making them sooner is risky.

Fixed price projects are often more expensive than they should be. The rigidity of these projects often causes scope to be too large. At the start stakeholders in the client-organization will specify many requirements and features they probably won’t need ‘just in case’. They know it will be hard to get them in later. Fixed price projects also force the supplier to make very accurate estimates at the start of a project to reduce risk, the time spent creating these estimates are later often added to the price of the project. Suppliers often also add a fee for the extra risk they have to carry, a former employer of mine added 20% to fixed price contracts and seeing how much trouble fixed price was for them this was actually low.

Instead of aligning interests of the client and the supplier fixed price projects artificially opposes them. The client will want to stretch the scope agreed in the contract as wide as possible while the supplier will want to do as little work as possible. This often prevents good communication while increasing risk. I have yet to see a fixed price project where the client delivered an on-site-customer and I don’t think this is a coincidence. Fixed price pushes stakeholders into roles that are not beneficial to the success of a project.

Fixed price and BDUF
I’ve heard more than once that these problems make fixed price problems especially to do in an agile way. A potential employer once told me when I asked him if they used any agile practices “We do fixed price so we can’t do agile”. When you look at the downsides of fixed price projects this seems logical. Fixed price has problems with some of the core agile values like flexibility and communication. The focus on up-front estimation also seems to point to big design up front or BDUF.

Unfortunately more traditional approaches won’t help us either. Big design up front will not help with estimations. We need accurate estimations before the project has even started. There hasn’t been time to gather anything resembling a complete set of requirements. Traditional estimation techniques like function point analysis are only as good as the data you put into them and unfortunately the data isn’t very good at the start of a project. Problems with flexibility and communication are only hidden, not solved.

Fixed price has some big problems. Not only for the supplier as many clients seem to believe but also for clients. These problems don’t go away by giving in to the traditional big design up front methodologies although some of the problems with flexibility and communication will be less visible. In the next article I want to look at how you’d do fixed price in a more agile way. This won’t solve the problems but it will give some tools to see them coming and to make it easier to switch contracts during a project.

The 7 Software Development Wastes – Lean series Part 3 – Extra Processing

Introduction

Waste is a killer in any organization. But if you don’t know what to look for then you’re probably not going to be able to find waste and minimize it or eradicate it. This series primarily deals with explicitly defining the 7 wastes in software development so that you can start thinking about how this may affect your ability to remain as productive as you can be. This can make a significant difference to your company’s bottom line.

As reference, the first two wastes can be found in my previous blog posts here: In-Process Inventory/Partially done work and Over Production/Extra Features.

#3 – Extra processing

Extra processing in Manufacturing is the 3rd waste and this can be equated to Relearning or Rework in Software development. The time we spend relearning things we once knew or having to rework the same feature due to poor code quality can be significant in many organizations, ergo, it’s important to be mindful of this. It’s probably useful to list examples that would cause extra processing in software development

Undocumented code

If your developers don’t document the code properly then later on when you’re either fixing the code or making changes to the code due to new feature requirements, your developers (even the original author of code) are going to need to relearn why the code was written that way in the first place. Taking the time to document your code up front while it’s fresh in your mind can therefore save the company time and money.

Poor planning

Some project managers assign work to developers in a haphazard manner. Although you do want to ensure there’s knowledge overlap on your team, if you keep switching developers on the same features, each time a new developer takes over, he is going to have to relearn what the original developer already knows.

Poor quality

The earlier you find bugs and fix them the less costly it is for the organization. The most expensive scenario is when you find problems once they’re already deployed to production. Partly this is due to the fact that the developer now has to reacquaint himself/herself with the code again. So if the developer spends the time upfront to write extensive unit tests, if the team takes the time to define proper acceptance test criteria then the chances of having to rework the code and worse yet relearn the code diminishes significantly

Task switching

The more things you throw at a developer at the same time the longer things take to get done. Task switching beyond 2 tasks is highly inefficient as the developer has to more often than not relearn or re-acquaint himself with the task context each time he switches back.

Poor communication and recording of knowledge

Today, with the proliferation of technology, there’s no excuse for poor communication. Wiki’s provide a great way for developers to record knowledge about their findings. And search makes it really easy to find information. I am not advocating heave documentation here. I am suggesting short summaries of information on key decisions you made during development so that if someone else has to take over or you have to come back to that code in the future, the relearning curve is not that great.

Stay tuned for next weeks blog post in this series which will cover the 4th waste – the equivalent of “Transportation” in manufacturing processes.

Hacking code ownership

Two weeks ago I wrote an article about the virtues of sharing code in teams. Just to confuse you, this week I want to tell about a situation where code ownership actually helped me introduce a couple of agile engineering practices in a team I worked with some time years ago. Here’s the story. Facts, names and situations have been twisted a bit to ensure the client’s privacy.

The project
When I got started on this project the team had already been working on the code for some time. They didn’t have strict rules about code ownership but most developers had started on their own piece of code at the beginning of the project and had kept on working in the code they knew as the project went on. When I joined the team everyone had a well defined layer of the application to call their own. I got to work on the UI layer.

On previous assignments I had tried introducing TDD and pair programming. But I often ran into problems. Maintaining a set of tests is hard when your co-workers change your code and don’t fix tests when they break. You end up spending extra time fixing the tests other people break and it’s hard to convince people to do TDD when it’s costing you so much work.

Introducing TDD
But now things were different, no-one touched code. I was free to start writing unit-tests. At first this wasn’t easy. UI’s are never easy to test. But I started slowly refactoring all the screens I worked on to extract the presentation logic to separate presenter classes that I could test. Slowly my test coverage got better and slowly my code got cleaner making it easier for me to implement new features.

Introducing Pair programming
Code-ownership also created extra dependencies within the team. Changes in the code usually went right across the artificial boundaries. This created an ideal opportunity for me to introduce pair programming. If someone wanted me to implement something I’d invite them to sit behind the computer with me to help me figure out what I needed to build. I was actually kinda surprised how quickly people picked up this habit.

Because I actually had the chance to be successful with TDD and because pair programming gave me the opportunity to show this off to my team members they started picking this up. The guy responsible for the data-layer actually had been doing TDD on his personal projects but had had difficulty doing it over here because most of his code was tightly bound to the database. I helped him separate the logic he wanted to test from the pure data-access code and helped him get started using Rhino-Mocks, a .Net mocking framework.

Because my contract ended I didn’t see that project through to the end. I heard they started sharing code more when they got more comfortable with each others code through pair programming and they were experimenting with continuous integration now.

The retrospective
Lets do a project ‘retrospective’ and see what we can learn. Should or shouldn’t we get rid of code ownership? I think we should, eventually, but whenever you get rid of something you should think of the value it brings first.

In his book Working Effectively with Legacy code Michael Feathers talks about using an anti corruption layer to shield our newly written code from getting corrupted by bad design decisions made in legacy parts of our system. These layers are a temporary fix, as the old code is upgraded to new quality standards, piece by piece it slowly move and eventually become unnecessary. Sometimes we also need an anti corruption layer in our teams and ironically code ownership, a practice I criticised in my article two weeks ago, gives us the means to implement such a layer.

Another thing that surprised me here was how easy it was to get people to do agile practices when they were introduced at the right pace in the right order. ‘Refactoring’ a team to agile instead of trying to become agile in one big leap actually has helped me more often. When you take small steps and be careful to take the step that brings the most value every time acceptance is easier and faster and that will make sure people will keep on improving even when you’re gone.

Agile is Dead

The agile revolution has won. There is no doubt about it. No longer are its evangelists the pioneers who blaze new trails and chart the unknown, like they used to a couple of years ago. Agile principles, methodology, tool sets, even agile lingo, have become mainstream. Frankly, everybody and their dog is “doing agile” these days. Of course, there are pockets of resistance in the more backward (or “conservative” to be more polite) companies, but they are destined to eventually see the error of their ways and switch over. Even the most pointy haired of pointy haired bosses know that if they don’t learn, accept and embrace agile methodologies, they are doomed to very soon become everybody’s laughing stock, not to mention their careers are going to go down the drain.

And that my friends, is agile movement’s doom. The revolution is over. The establishment has struck back, as it usually does, using the strategy it always chooses – assimilation, dilution, extermination.

These days, you can call everything “agile”, even the most ridiculously non-agile concepts and get away with it (“agile RUP”, anybody?). See, the problem is that the PHBs that decided that they absolutely have to “go agile”, at the same time decided that they don’t really want or need to change their habits – instead they will just re-dress and rename their usual processes and culture. Which obviously means two things:

  • buying yourself an expensive, yet “agile” tool set, so as to justify inflated budgets
  • getting a bunch of costly certification or two, so that nobody higher up in the food chain can claim they are not “agile”

I have noticed quite a few comments on this site, typically posted by the “foot soldier” developers, that “this whole ‘agile’ thing is same old, same old. All it does is bring us more meetings, more reporting, more micro management”. How typical. Agile processes are being introduced in companies without paying attention to the absolute fundamental need for a culture shift – it really is “same old, same old”, it is just dressed funny. The end result is that teams mindlessly follow “agile” process steps, because the expensive, certified consultant told them to, or a costly tool, mandated by the upper management, forces them to. And they fill out more and more paperwork — even more than they used to before they went agile. Unit test reports (usually hand-filled in Excel). Ridiculously detailed time tracking. Insanely elaborate acceptance criteria for user stories. Torture of daily standups (an hour of it every day, with mandatory report-out to management). It is all in there. There is not a trace of principles of communication, respect, trust, self-organization left in such teams (it has never been there in the first place), they do not evolve to becoming creative and effective, but they are still “agile” and they can prove it! They are certified!

And quite honestly, we have brought this upon us ourselves. Instead of concentrating on spreading the word about the principles – which is hard, because traditionally-bred teams and managers have trouble understanding, not to mention converting – the agile community has concentrated on proliferating certifications. Certified Scrum Master? Scrum Practitioner? Scrum Trainer? Grand Klan Master? Chief Lizard Wrangler? Cappo di Tutti Cappi? Sure, why not, just shell our a bunch of monies and sit there for two days, pretending to listen to some bloke that already is certified by us. Admit it – it is a Ponzi scheme. There is no proof whatsoever that becoming “certified” makes you a better developer or project manager – but it does mean that the thing is easier to sell to top level management. ISO certifications and CMMI server the same purpose – improving the chances of selling consulting services to managers. And of selling software to customers (“gee, they must be good, they have ISO 9001 and CMMI 5, let’s buy from them”).

It does not stop there. Soon, agile is going to have a “maturity model” of its own – backed by IBM, no less. Who cares if it is worthless — it is from IBM, so it is “professional”, right? I wouldn’t be surprised if the next move is for some big player to simply go out and acquire trademarks to all relevant agile processes – “Scrum(TM) and you can only use it under license form Microsoft”? “XP(TM) brought to you by IBM (and we better not catch you using it without paying us)”. Why not? What’s to stop them?

The revolution is over. It is time for a new one.

Or is it? I invite you to a discussion. Convince me that I am wrong.

The 7 Sofware Development Wastes – Part 2

Introduction

I have been writing for AgileSoftwareDevelopment.com for nearly 6 months now and was quite shocked to see how many folks read last week’s blog post. I can only assume from the response that there is a lot of interest in Lean. Which in my opinion is a really good thing as we can learn tremendous lessons from this philosophy. I wish I knew about Lean earlier in my career – I think it could have made the world of difference. Lean offers such practical advice on what to look for in order to make the process better. I think just knowing about these aspects of lean will help you day to day, help you make better decisions, make you a more effective player on a software development team.

Last week I covered the first manufacturing waste In- Process Inventory corresponding to partially done work in software development.

This week I will discuss the 2nd Waste:

Over – Production

Over production in manufacturing amounts to production of goods that are not required or consumed immediately by a customer. This is very costly in a manufacturing plant. Inventory can become stale, superceded, damaged etc.

In the software world, over-production is the same as excess features, features not really needed by customers. My understanding is that there is evidence to backup the fact that over 60% of features in software are ever used.

The reason for excess features in software development is a direct result of traditional waterfall methods. Product managers got one shot to get all the requirements out a the beginning of the 18 month project. So we forced them to think of absolutely everything. As a result more huge amounts of time and energy went into developing software that would never actually get used.

So why is it wasteful?

Well firstly, there’s the direct costs of development but there’s all the indirect costs like adding complexity to the codebase, introducing unnecessary bugs, contributing to sluggish performance of the application and the list goes on.

What I love about Scrum is that it’s designed to focus the organization on delivering just the most important things, just-in-time. So I’d argue that Scrum really helps mitigate the risk of over-production.

Sum it up

Be mindful of this waste, keep it at the back of your mind at all times, ask yourself “is this really necessary?”

A lean codebase is an agile codebase. It gives the product owner and stakeholders options – options to change direction at a moments notice, to respond to market conditions to be more strategic.

Next week I’ll cover the 3rd waste – Extra Processing