In large organisations, even within the digital by default public sector, there remains unfamiliarity and uncertainty around agile. Our experience suggests that some of the largest organisations within the public sector continue to have difficulty adopting agile, preferring a traditional waterfall or hybrid agile-waterfall approach. What are the reasons behind the agile uncertainty and how do we address those?

Recent Beginnings

Agile has come a long way since the manifesto was written in 2001. Many of us are familiar with the principles of the agile manifesto, focusing on what users really need, producing working software and continually responding to change. As you might expect, some were quick to respond to this new approach to project delivery, and not too far behind, the public sector was also waking up to what would soon become the agile revolution.

Digital Revolution in Government

In 2010, Martha Lane Fox (the UK Government’s Digital Champion) penned a letter to MP Francis Maude. In it, she discussed the findings of her review of Directgov, talking about revolution not evolution, and a need for the Government to move to a service culture with a big focus on users.

Since that letter was written we’ve seen the Cabinet Office’s Government Digital Service (GDS) team formed to deliver digital transformation in government. Its purpose is to be a centre of excellence across digital and supporting other government departments with their own transformations.

The GDS team has been very open about what it does and how it operates. It created digital standards to focus the development of new online services on user needs, making informed decisions with real data, testing and iterating to improve, and making services consistent and accessible. Those standards are the basis for the GDS assessment that all new public facing transactional digital services must pass.

Augmenting Private and Public Sectors

Public sector organisations open up opportunities via the G-Cloud and DOS frameworks for private sector companies to design, deliver, host and audit new digital services. The requirements for these opportunities will often state the need for agile but there often remains the requirement for elements of delivery, reporting, fixing scope and budgets that are more aligned with traditional waterfall approaches.

The unfamiliarity of agile and the myths that surround it lead to requests that seek to blend the old with the new in a way that’s not complimentary, regardless of the many agile delivery successes in all sectors. A misunderstanding and lack of knowledge of agile leads to myths such as there is no plan, there’s no certainty, it’s chaotic and ill-disciplined, there’s no governance and no documentation. This leads to a lack of appetite for agile adoption.

Debunking the Planning Myth

Myths are often created by those with limited understanding or bad experiences of agile. Those more familiar with agile delivery would debunk the “agile doesn’t do planning” myth, arguing that there is more planning, and that it features continuously throughout the lifecycle of every agile project.

With traditional waterfall delivery, requirements are documented in full, there is a lot of upfront planning, gantt charts are created to show how the project will be delivered then this is all handed over the fence to a project team whose job it is to deliver by the date shown on the plan. This thoroughly planned approach creates an unfounded-confidence and false sense of security. As the project progresses, cracks quickly begin to show and the lack of flexibility in the waterfall approach beings to affect the deliverables.

Despite the introduction of the GDS team, its standards, open communication and success stories, many public sector organisations continue to struggle to accept and adopt agile as the evolved, more effective way of delivering projects. They continue to try to improve and automate workflow and processes using antiquated systems. These methods make the interfaces between digital service users and government organisations difficult and they present a barrier to change. Despite the emergence of new efficient and transforming technologies, many government organisations continue to nurture familiar but inefficient ways of working and delivering services.

As experienced agile experts and evangelists, able to apply the frameworks pragmatically, it’s our job to support the public sector in its agile transformation. The number of completed Government digital transformation projects in the last 3 years demonstrates that subject matter complexity is not a barrier to success. Unless there is strong product ownership and the skills and experience to deliver with Agile effectively, there will always be the reluctance to leave the security blanket of waterfall. The public sector provides digital services that affect us all so we need to work together in the transformation and help make our public services better, more accessible and more able to adapt.

Objectives & Requirements

As with any project, regardless of the approach, we capture objectives for our agile projects. The objectives help us maintain the direction of our projects and influence our decisions.

Similarly, our agile projects also define requirements, just not in the way you might be familiar with. Agile requirements are written from the point of view of the user. While the overall project objective will be focussed on the business, to make the project work we need to satisfy the needs of our users, help them achieve what they need through the service we’re developing. If we provide what the users want and need, if we understand their requirements and meet their goals, then our service stands a good chance of delivering purpose, adding value and being a success.

Project Planning & Change Management

Contrary to what some people believe, planning plays a massive part in making agile work. We plan our future activities at a high level and we don’t plan the lower level details until they near the development schedule. We do this so that we can make decisions as late as possible, once we have the benefit of the knowledge we’ve gained through the delivery up to that point.

There are different ways of documenting and communicating project plans. How that’s done is something you agree with your supplier. Typically, the requirements are listed in priority order in the product backlog and this describes the order in which the project will deliver. The benefit of this approach is the simplicity with which the order can be changed.

The important thing is that you maintain visibility of the plan and the requirements for the whole project.

Consider how difficult and discouraging it can be to make changes in a traditional project, the processes we need to go through to assess the impact and cost of the change and the effect it has on the schedule. With agile, it’s a far simpler process, one that encourages change for the sake of delivering value and making the service relevant to the user.

Consider the benefits of being able to make changes very simply on a daily basis, adding, amending or removing requirements and re-prioritising to fit your needs and the drivers behind the project.

Reporting, Communicating & Governance

Agile is very open and inclusive when it comes to reporting and communicating. Knowing what’s happened, what’s being worked on now and what’s planned for the future is a mandatory part of agile communication.

Too many times I’ve heard of people being suspicious of new projects. Their experience is of projects being developed behind closed doors and them not hearing anything until the big reveal or hearing nothing at all because the project has failed.

Unlike traditional projects, there is no big reveal approach with delivery. Instead, agile involves the client every day, describing what’s being worked on. At the end of each iteration we demonstrate the progress, the work that has been completed and we encourage clients to involve their wider business in those demonstrations.

The open and transparent communication style of agile supports project governance. Stakeholders actually see what’s been done at the end of every sprint (usually 2 weeks) and get the chance to play with the completed software, to ask questions and satisfy themselves that a real service is being developed and that the progress is as reported.

Continuous Learning & Improvement

For me, the fundamental benefit of any agile project is the ability to continuously learn, adapt and improve. We deliver in short iterations (sprints), usually two weeks. At the end of each sprint we review the completed work and how it was delivered. What we’re looking for is whether it meets the requirements, whether the user is satisfied with what’s been delivered and how we can improve the ways in which we manage the subsequent sprints.

Agile delivery teams maintain a high and constant rate of delivery rather than the peaks and troughs we often see in waterfall. That high delivery rate is one the team seeks to improve over time, increasing efficiency, improving quality and ways of working. Customers new to agile are quick to see the benefits of the team meeting briefly at the end of a sprint to agree how they can improve. They’re looking for ways to increase the value for money our clients receive.

Supporting the Agile Transformation

Some organisations choose to dive head-first into agile while others have a more cautious approach to change. Whatever approach you take it’s important to have the necessary support behind you to make the transition.

Agile frameworks such as Scrum and Kanban are straightforward to understand in principle but can be challenging to apply in a commercial environment. A lack of experience is often why agile projects become unstuck, agile then becomes the reason for the failure, making it almost impossible to convince management to try a second time.

Choosing the right partner to help with your transformation is vital. Many organisations state technical ability and price as the primary scoring criteria when assessing proposals. While these are important factors, greater emphasis should be placed on working relationships, deciding whether or not you feel you could work closely with that partner company.

With agile there is an expectation and insistence that you work closely together as one team. It’s vital that the organisation you choose to work with is more than a just a good technical fit. You need an extension of your team, one that will challenge your thinking, help you maintain focus, provide coaching and support to help you maintain the direction of the delivery.

Next time you’re writing your requirements, consider any delivery concerns you have or bad experiences and ask the vendors to address those specifically. Being able to provide real evidence will help to show the depth of their experience, their ability to apply that to your situation and it’ll give you an idea of how you’d feel tackling those issues together with that particular vendor.

GDS Support

Because many public sector organisations do not have the knowledge or experience to understand how agile their implementation is, how effectively the vendor has implemented the practices and measures, and the way in which user feedback has affected what’s been delivered, projects are subjected to Digital Service Standard assessments.

This standard is a set of 18 criteria to help government create and run good digital services. All public facing transactional services must meet the standard. It’s used by departments and the Government Digital Service to check whether a service is good enough for public use.

The assessment, carried out independently, provides assurance to the buyer that their supplier has provided a service that meets the standards expected.

Evolution or Revolution?

As I mentioned earlier in this post, there is a digital revolution happening within government. However, a revolution is often too much for some organisations to contemplate, at least until they have some experience and understanding of what they’re dealing with.

Whether you’re prepared for revolution or want to take a more risk-averse evolutionary approach is down to your appetite for change and risk.

Whatever your appetite for change, you’ll be moving away from the old approach of gathering requirements, designing, developing, testing and delivering, and only finding out at that point whether your new service works, whether the users accept it. The new agile approach will perform a little of each of those traditional steps each week. You’ll be delivering increments, receiving feedback, assessing that feedback and prioritising changes, seeking to improve what you deliver in the next and subsequent sprints.

The project will progress through a number of stages before being live. Projects begin with a discovery phase, scoping out the project, creating a shared understanding of the service. The alpha phase develops an early proof of concept so that the team can see mistakes early, fail fast and improve faster. The beta phase is when the service becomes real, when we start to talk about security and data retention, when we prove that the service will work. This structured, phased approach protects your project, gaining approval from the team and stakeholders at the end of each phase before agreement is reached to move to the next phase.

Agile can be tricky to implement properly but you can manage this risk by working with a supplier who understands and has experience of agile and transformation. The right supplier can help with training and coaching to support your transition and help you benefit from the process, watching your service grow and improve as you deliver, test and iterate.

One of the critical success factors is management buy-in. The traditional top-down approach to management will not work here. To make agile transformation work it’s imperative that management fully supports the shift, that they assign the right people with the authority to make change and that they’re there to fully support and coach those people rather than providing traditional top-down management.

Collaborate for Success

Any decision to change the way you deliver will generate risk and uncertainty, be it financial, operational or reputational.

Agile has opened up many opportunities to organisations across the sectors, enabling them to make incremental change where previously those changes were deemed too big, too costly and too slow to implement with a “big-bang” approach.

Adjusting your procurement process to help you identify the right agile partner is an important step in making the change. Asking for more examples of project delivery with agile, using scenarios rather than strictly scored responses, giving partners more room to open up about their experiences, about what they think is important and about how their experiences would support your transformation will help you understand more about their specific skills and whether or not they are the right supportive coaching partner for you.

What you will experience with the right partner is the increased level of collaboration. There is much emphasis on talking, discussing the issues, communicating them well and making sure everyone on the team understands them. Collaborating, iterating and validating is a real opportunity for your team to innovate and experiment.

Successful projects need a good, solid foundation. The right partner will help you set up the project properly from the start, helping people to understand their roles and responsibilities, and putting the necessary governance in place. With the right support you’ll begin to understand, discuss and use the knowledge you’ve gathered. You’ll begin to see the benefits and value of collaboration and user focused delivery that agile encourages and be one step closer and to being the latest public sector digital success story.