Over the years I have witnessed a number of organisations adopting agile - often starting with just one team. Moving from traditional waterfall to agile is a challenging and exciting journey that can uncover a lot of false assumptions and misunderstandings about agile during its course. One misconception I have often seen is around using story points for planning, forecasting and progress tracking. Long- and mid-term planning in agile can negatively impact teams, which is why I’d like to shed some light on it and talk about how the situation can be tackled in an agile way providing much better results for the project and the organisation itself.

I can use story points and velocity for planning and tracking - so why not?

Planning is done in many parts of organisations including budget planning, marketing plans, personal growth plans, recruitment plans and so forth. Those plans usually form part of a wider company strategy and involve costing, timing and ideally success criteria. Similarly, when deciding to conduct a certain project, decision makers want to know how much it will cost, what the value of the outcome is and what the return on investment will be. This is usually shown in a business case and it seems very sensible to ask those questions before someone makes a decision to invest a large sum on some project work. Taking this further, such a business plan creates expectations on time and scope. That in turn leads to someone trying to make the project adhere to those figures, progress is tracked and compared and here we are in the typical ‘hamster wheel’ of waterfall projects. One very good reason for going the agile route is the understanding that detailed plans on long time horizons are prone to failure. Successful waterfall projects define their success with hindsight; change requests and large contingencies are used to change scope and time expectations and what has been stated at the beginning is nearly never delivered in the original timeline with the expected quality, although we invested so much time and effort in detailing upfront all those requirements. Everyone in and around a project wants to be successful, so we define success in the right way and we all win. This is not deriding all the good work that can be done in a waterfall project, sometimes we do something great even in waterfall. But we are kidding ourselves about our ability to plan and execute accordingly and what is more important we could do so much better so easily.

Do better easily? How so?

The reason plans fail is the enormous level of uncertainty contained in any project. This is because when we do something for the first time, we cannot build on experience, often we do not even know the tools needed at the start or if the outcome is desirable at all.

• We need to learn how to do it
• We may not even know what we need to produce
• We may need skills we don’t have
• Teams change over time which leads to a knowledge drain

We can do much better if we accept this, embrace it and start on a journey that is more like a deep sea exploration instead of acting as if we are building a module house to a known blueprint. There are many things in the lean toolbox to help us navigate, e.g. spikes to explore, prototypes and MVPs to get tangible results quickly, user testing to check desirability, pivoting to change direction and much more.

Wait, why do agile teams estimate then?

Teams do indeed estimate, one very popular technique is to use story points for sprint backlogs and T-shirt sizing on a high level for the product backlog. And the team does use this to plan. But let’s see why: First the product backlog estimates are to inform the Product Owner defining priorities on the product backlog. She does not plan in time but creates a sequence of importance where effort of a task is relevant. Next. The team estimates rather detailed but they use it only to fill one sprint. That is a very short time horizon. Estimating their own work is very useful for teams not so much for planning but for learning: when we estimate we need to understand, we ask the right questions, break stories down into the right level of detail and make sure we have the information needed to work on it. Planning in time is the least of the concerns. The commitment of the team to do certain stories in a specific timeframe is a conscious decision that strongly supports a sense of responsibility and motivation. That’s why teams estimate in agile.

What about velocity?

Together with a burndown chart velocity can help identify early warning signs and supports a better understanding of what is going wrong or well in a sprint. Velocity is a bad tool for tracking and forecasting because the absolute figure can change quickly and radically, e.g. when the composition of the team is changed, new topics are started, tooling needs to be changed and much more. Alright. But I can still use the estimates for my progress tracking and high level estimates for long-term plans - on top of what the team does. We can do the maths but you won’t get anything out of it than the old illusions. Long-term planning works no better in agile than in waterfall. If we use the agility in the approach we will change direction and scope all the time, making the plans useless. Even worse is the effect on the teams: the above mentioned feeling of responsibility and motivation is destroyed: The team has never bought into the long-term plan and cannot, due to the lack of information. This often leads to gamification and a ‘cat and mouse’ game between teams and those who try to track progress and report on it.

Man, I cannot spend x-million £ without knowing what I get for it.

True, that would be folly. But instead we can use a lean approach: start with a vision, define the very first step in the direction of your vision and verify the result with user testing. Use the feedback to learn and define the next build-measure-learn cycle. These cycles can be very short, maybe 2-3 sprints at the beginning and only one later. This means you need only enough budget for these short cycles, can present outcomes in terms of a tangible product outcome, but also in terms of learning, and justify the money spend easily.

So, financing should be agile too?

Yes, agility is a feature of an organisation not just a project team. A company can of course set aside a certain amount of money e.g. for new products and use one or more agile teams to work on these. Agile means we learn what best to pursue, we can change direction quickly, re-prioritise scope based on what we learned and even stop projects when they don’t seem useful based on feedback. We won’t know exactly what product we will get at the end of the year, but we do know we will get the best we can given the ideas, people and time we invested. We avoid spending lots of money on something potentially nobody wants, reduce time to market and hence get to the return on investment much quicker.

But what about other time dependencies like marketing? I need to plan that !

Marketing is a good example for a time dependency, we may want to tell our target group what amazing things we are going to give them, create anticipation and maybe even a hype to increase our future market coverage. There are prominent lean proponents that have combined a lean approach with early marketing, e.g. Monzo, who marketed early on with a strong, simple vision and gave away only limited access to an MVP and has later provided a continuous stream of small improvements leading to their vision. People love the participation, endure failures with a smile knowing they are part of an exclusive club of beta users for something that could very well be the future of banking. We should not be shy about marketing early versions if we do it in the right way. We can get enormous buy-in from our market, beta testers who want to influence a product for the best and geeks that love being on the bleeding edge of whatever.

Conclusion

Planning in itself is not evil but we need to make sure we plan on the right time horizons and at the right level of detail. We should not use team estimates to recreate a whip as it undermines the inherent level of responsibility and commitment present in good agile teams. Finance as well as other functions will be involved in being agile - agility is an organisational attribute, not something limited only to project teams.