The Single Biggest Mistake Organizations Make When Adopting Agile

I know what you are thinking, with so many ways that organizations can undermine their steps toward agility, how could you pick just one single biggest mistake?

Well, I keep bumping into this mistake with organizations and increasingly I think it is the single biggest mistake. It is the quickest way to undermine the benefits of agile ways of working. That mistake is organizing around projects, rather than organizing around teams.

Why is organizing around projects such a big problem?

Buckle up, this is going to take some explaining.

Why is Organizing Around Projects a Mistake When Adopting Agile?

#1 – It is Incredibly Inefficient and Disruptive to keep Starting new Projects

Imagine that you had a fast food restaurant that served burgers. As customers came in with their orders, you wouldn’t take the order and then turn around and ask your staff who wants to work on each burger right? That would be crazy. Yet in most organizations today, even ones claiming to leverage agile ways of working, they do exactly that. They keep spinning up new projects to satisfy each new business request and then they ask or they assign people to the work.

Starting new projects is disruptive. Even if every person needed for the project is available, which they aren’t, the initiation and planning take time. And then the new team has to go through the unproductive stages of Tuckman’s stages of team development – Forming, Storming, and Norming – to get to the productive stage of Performing.

This starting and stopping is inefficient.

#2 – It is Impossible to Determine the Optimal Number of Simultaneous Projects

In addition to the inefficiency of starting and stopping all those projects, we also need to recognize that we don’t know if we have the people. It is nearly impossible to determine the optimal number of projects you can run simultaneously in your organization.

Sure, there are portfolio management and resource management tools that have mostly replaced the excel spreadsheets that were traditionally used for resource management. And in theory, those tools can tell you how many projects the people in your organization can support.

These tools fall short in telling you the capacity that will maximize throughput and productivity. That is because of the knock-on effects of assigning people to multiple projects, over-assigning people, and hidden work.

However, as we all know, we can keep on starting more and more projects if we simply re-assign the same people over and over again on those projects. So in theory, we can assign people fractionally to more and more projects. Each project will take longer – a lot longer – but we can continue to start projects. Never mind all the overhead from managing all these projects and reporting on them.

The real constraint in most organizations is capacity or the number of people you can bring to bear to get work done. It turns out that the number of people is relatively inelastic, even in organizations that use contract labor. Sure you can toss a big project to Accenture but that is still going to require you to leverage people in your organization from purchasing, management, quality assurance and project management. If you are looking for low-cost labor overseas you be prepared to commit even more of your people to that effort.

So if we accept the argument that capacity is somewhat inelastic, let’s look at the relationship between capacity and projects underway.

For a given capacity, the more projects that are underway, the less progress will be made on each project. Or as a CIO at one of my clients succinctly stated recently, “I’d rather have 5 things going 100 miles per hour than 100 things going 5 miles per hour”.

If we assume a linear relationship between the number of projects underway and speed, for a given capacity, you might plot the relationship as shown:

Time to Complete Projects vs Projects Underway Theoretical

In theory, taking on more projects means a linear decrease in time that can be given to each one. And therefore, all projects are going to take longer.

Unfortunately, it turns out the relationship between the two variables is not linear. In fact, as we add more projects, time to complete all projects goes up in an exponential fashion. That is because of the overhead of context switching and as well as other factors.

In Quality Software Management: Systems Thinking, Gerald Weinberg proposed a rule of thumb to calculate the waste caused by context switching. He wrote that as you add more tasks or projects or concurrent tasks, the amount of waste is about 20% for each concurrent activity. This is illustrated in the diagram below.

Cost of Context Switching

His chart shows that the more context switching an individual does, the more of their time is wasted in context switching.

Students in my classes claim to be assigned to 3 or 4 or more projects at the same time. They spend most of their time attending daily meetings for those projects explaining that they don’t get any of their tasks done because they spend all their time in meetings. [See Stop Assigning People to Multiple Teams]

So if we zoom back out to the organization level, and incorporate Weinberg’s thinking, we can reflect the cost of context switching and show overall productivity like the chart below. There will be a theoretical point beyond which adding more projects decreases individual productivity to nothing more than overhead. This is the point where people in my classes tell me that they have so many simultaneous projects that they spend all their time going to meetings to talk about the tasks not getting done.

Time to Complete Projects vs Projects Underway Reality

In theory, there is a sweet spot where overall productivity is maximized based on the overall number of concurrent projects. You are very likely beyond the point of maximum productivity. And you don’t have a good way to know that because it is impossible to determine your capacity.

But even if you did know the optimal number of projects, I contend you would still take on more projects than you should. Here is why.

#3 – You Are Taking On More Projects Than Optimal

Even if you knew the optimal of projects, I could predict with confidence that you would take on more projects than that optimal number. I can say that without being familiar with your business. And it is not because you don’t know what is optimal.

You take on more than optimal for two reasons – hope and the inability to say no.

By hope, I mean that you use hope as a planning tool. We hope to achieve these by the end of the year. You hope that by setting aggressive deadlines the teams will somehow stretch to meet those goals. You hope that by setting 20 top priorities instead of just 10, that the team might actually hit 10. You hope that by assigning your best person – Jen – to your 5 top projects, that she will somehow find a way to deliver on all of them.

And you cannot say no. Cannot say no to just one more project. Cannot tell a sponsor that the pet project they are requesting cannot be done.

Just one more wafer-thin mint, as Monty Python fans would say.

The inability to say no is so common that at one local organization where I consult, they began calling their stage-gate “Go / Go” reviews instead of “Go / Don’t Go”.

Our inability to say no is like the desire to push things to 11 on a scale that should only go to 10:

Dial up to 11

What is not happening is limiting Work in Process.

#4 – As a Result, Project Delivery is Unpredictable

If you are using projects, I would wager that your delivery is unpredictable. Or I should say, I can predict that they will be late and overbudget. And the statistics are behind me on this.

On-time and on-budget delivery has been the bugaboo of technology projects for years. The business model of the Standish Group since 1994 has been based on studying and publishing project success and failure rates. According to the Standish Group, only one in three projects is delivered on time and on budget. Though Agile “projects” fare better, the results are still dismal.

If you are using projects, you have more than the optimal number underway and the delivery of those projects is unpredictable. It is inefficient and unproductive.

Don’t Make that Mistake When Adopting Agile – Do This Instead

This project paradigm is hard to break. There are many advocates for doing away with projects completely as in the #noprojects movement. You can read about #noprojects in this InfoQ article: If you Need a Project You Have Already Failed.

I recently spoke with a client who said they were using agile and yet still using projects. They have some projects that are just one sprint long (2 weeks). Yes you read that correctly, a project completed in just one 2-week sprint. They disbanded the team after just one sprint. Wow, way to take an agile concept like iterations and bastardize it.

Adopting Agile while keeping projects is a half-measure that is going to limit the benefits of agility. So rather than going halfway, improve your success with agile ways of working by doing the following:

#1 – Organize around Long-standing Stable Teams

Capacity is really hard to gauge unless you organize into teams and then forecast based on team capacity. This forecasting is better than the project approach where we plan capacity by person or ignore capacity altogether. Both of these approaches miss the mark and wind up with overcommitments, hidden bottlenecks, and phantom delays.

Instead, organize around teams.

Put one person on just one team.

Yes, take the people that you have and form long-standing teams. Do this once and consider adjusting their membership after a year. Just set them up and leave them alone.

This provides a mechanism for accurately forecasting work.

And for taking on work to capacity.

And it avoids the problem that projects carry:

  • Inefficiency of starting and stopping
  • Cost of Forming Storming and Norming

#2 – Teach Teams to Limit WIP and Work to Capacity

Who is in the best position to judge how much work should be done? Yes, the teams. With proper training, teams can properly manage their backlogs, break work down into appropriately sized chunks, plan based on their capacity, and optimize their throughput. It may sound complicated but it is actually quite simple.

This avoids the problem of the over-assignment of work. Let the people closest to the work determine how much is optimal.

#3 – Let Teams PULL Work Based on Their Capacity

Avoid the entire over-assignment of work to people by stop assigning work. Yes, you read that correctly.

Use agile Portfolio Management techniques that prioritize work, limit work in process and provide queues for teams to pull work into their backlogs.

Say no at the portfolio level.

You Probably Will Ignore This

If you’ve read this far, you are probably already doing the things above. Congratulations.

Sadly, those that did not read this far are probably not going to change. Moving from the project metaphor would mean giving up a sense of security and predictability. People don’t want to give up those single throats to choke that they get with project managers. And it is too hard. It is likely you don’t really get agile anyway and are unwilling to learn and advocate for it. You think that using Agile in Name Only is enough to keep the brass off your backs without really changing.

Good luck with that.

This article originally appeared here and has been republished with permission