As a software consultancy, we’re more than a little familiar with Agile. It’s been the driver behind many of our successful projects of varying size and complexity. While it doesn’t deliver projects and resolve problems on its own, it has provided our teams with the necessary controls and tools to deliver some of our more challenging projects.

Organisational Rigidity

Many of our clients are large, multinational corporations with their own ways of working and those ways are rarely Agile. Even as a relatively big software consultancy it can prove difficult to introduce new ways of thinking, different ways of working, if we’re to deliver our projects within reasonable timescales. Agile transformation within large organisations requires the time, tenacity and patience that is rarely available. Simple project agility is a luxury we don’t often have.

However, the restrictions imposed by dated, ingrained, inefficient, organisation-wide delivery approaches is a common problem that we’re familiar with. Because many clients continue to deliver their own projects using increasingly antiquated project approaches we continue to see the same old problems reoccurring. Such problems include cumbersome change controls, unnecessary documentation that attempts to guess everything the project will ever deliver, and detailed planning of the entire project before development can begin. Flexibility does exist but it creaks, it groans and is generally reluctant and discouraging.

Whether it’s a reluctance to move from their current ways of working or simply a lack of knowledge and understanding about modern delivery methods and agile transformation, it’s our job to support the project. We need to bring news ways of working and thinking and help steer the delivery, to respond to change and make informed decisions while maintaining control, communication, direction and the confidence of users and stakeholders. It provides us with the opportunity to demonstrate more flexible and effective ways of working and the importance of continuous improvement.

For some time now we’ve been working with a large financial organisation on a number of projects to design, build and implement new digital services. It’s a vast, global organisation and predictably waterfall. The challenge for us has been to deliver the projects in the way we know how, in a challenging, uncomplimentary environment with the added complication of fixed deliverables.

Exploring Requirements

To begin with, there were clear, high-level requirements from the client. We started the project with a discovery phase to analyse user and business requirements, and to explore the tech and make informed recommendations. This provided us with a solid foundation and reasoning on which to build. What we’d need to be prepared for would be the ever changing requirements and the many stakeholders, each with a financial investment and a personal interest in the project. It would prove to be a difficult balancing act for the Product Owner and we’d need to actively support them in order to protect the delivery.

Protecting the Team

Based on previous experience, we knew we’d need to ring-fence our team, to have them work autonomously to stand any chance of delivering effectively while avoiding distractions. To effectively ring-fence the team would mean restricting who had access to them, routing requests and queries initially through a lead developer.

The product owner carried the overall vision for the product. There were real user problems for us to solve. We were replacing an existing product so were able to build up a clear picture of what didn’t work and a clear list of user requirements. Introducing a small team of UX designers into our delivery team helped give a UX-focus to the project, gathering detailed requirements from users, defining the initial low-fidelity designs and taking these through several iterations. This approach introduced the users and the client to cycles of planning, developing and reviewing, something that was missing previously. It started as a natural process, one that the users understood and appreciated, and introduced iterative delivery without forcing the Agile tag.

As the project progressed, demonstrations by the Product Owner communicated the new features and potential of the new product, and many requests for additional features began to arrive. The shape of the product was changing and pressure on the team was quickly building.

Managing Complexity, Change and Risk

The product we were building was complex, integrated with numerous other systems and tools, entirely bespoke, affecting many users and had numerous stakeholders, requiring careful management, direction and communication. How we’d manage the team, focus their efforts and provide the right environment for them to thrive would be vital. One of the biggest risks to efficiency and velocity is context switching. Continually changing the priorities, not allowing the team to complete one task before moving them to another new priority, is the quickest way to waste development effort and increase the risk of not delivering on time.

A Time for Change

As our project was subjected to increasing demands we decided it was time to take stock and review how we were managing and delivering. It had become apparent that the team was working in a less than perfect environment that could have a negative impact on the work it produced, the morale of the team and therefore the relationship with the client. It was time to make changes.

Our engagement initially started with an augmented team. We provided technical expertise to the client to help design and develop a very specific, bespoke editorial and publishing platform. As the project grew in size and pace it became clear we needed to provide a larger technical team and to add a delivery manager in order to provide the necessary support to both the delivery team and the Product Owner.

Introducing Scrum

While the project was dealing with the increase in requests, we made the decision to implement Scrum. The reasoning behind this decision was to introduce the plan-do-act cycle we knew the Product Owner needed to regularly review the progress, to see working demonstrations and to adjust priorities and direction as necessary. Regular cycles help to develop habits. If we could influence good habits then it would prove beneficial to the project.

The introduction of Scrum didn’t have an immediate impact but we worked at it over the course of a month, making adjustments, in particular reducing the Scrum duration to one week. As the Product Owner was providing more demonstrations of our progress to the business, user feedback and requests for change were mounting. Shortening the development cycle meant that we’d reduce the risk of straying off-track, helping maintain the correct direction and increasing the velocity of the team by providing more focus, avoiding context switching and working more efficiently.

Setting Expectations

Prior to us implementing Scrum there was no formal planning or estimating in place beyond the high-level contracted deliverables. The avoidance of estimating led to incorrect assumptions being made about the level of effort required for some stories. A lack of planning led to there being no real platform for developers to discuss the technical approach to requirements. Both of these activities are key to exploring a detailed understanding of requirements and setting realistic expectations.

Although planning was initially viewed with suspicion by the client - as a pause in the development, a time when real work should be happening - it quickly proved its worth. The team demonstrated its deep knowledge of the tech and the infrastructure, seeking alternative resolutions, asking probing questions and challenging the Product Owner as well as each other. This helped gain the confidence of the Product Owner by helping them appreciate what was going on technically, the amount of effort required to implement some of the seemingly more simple requirements. Being able to experience the challenges faced by the team, helping talk through technical issues and offering alternative approaches or adjusting the requirements helped to develop more trust and a better working relationship. It was the development of this trust that helped the project turn a corner.

A Platform for Open Discussion

When there is trust between the Product Owner and the delivery team it leads to more open and honest conversations. Those conversations can quickly deal with issues because the team feels more able to express concerns, to raise risks early and to talk about challenges. An open dialogue helps us deal with problems by discussing alternative solutions and to trim any excess off the requirements when we’re trying to add as much value as possible into a short space of time. It creates a more healthy relationship and a better working environment.

It’s now 9 months since we began to implement our changes, moving to outcome based delivery and implementing Scrum. The client was understandably nervous about us implementing a change in approach mid-project but the decision has paid off. It continues to be a busy project but it has a more structured feel to it now. We’ve maintained the weekly Scrum cycle, starting the week with planning and finishing with a demo. The velocity of the team is high, the quality of the output is good, expectations are based on the outcome of thorough discussions and the general pressure to deliver has reduced.

The Product Owner remains very close to the team and to the detail of those stories currently in progress. Careful management of stakeholders means that the Product Owner needs to keep a close eye on what’s happening day-to-day, reviewing progress regularly throughout the week and understanding the finer details of what’s being developed. This provides the team with another feedback loop and helps with efficiency since we’re reviewing so often.

Because of the complexity of what’s being developed, the system interacts with many other existing systems, tools and processes. The team is therefore required to interact with the client’s teams responsible for those other systems and has managed to retain its autonomy, keeping our own project separate from others, and keeping distractions to a minimum.

Communication is Key

Throughout the life of the project there have been ups and downs, successes and challenges, but we’ve maintained a good relationship with the client. During the tough times we’ve kept an open and honest dialogue, we’ve had the difficult conversations and we’ve made changes that have paid off. There was no doubt that we had a highly skilled and experienced team in place and a Product Owner with a clear vision for the product. Nobody was at fault for the difficult times. They were due to the environment we were working in and the challenges of working at scale and with pressure.

Overall, communication was the key to success here. We had the necessary communication channels in place to enable the client to raise concerns early. The team was able to discuss and work through the problems with the client. We then had someone in place to continue monitoring and managing the situation with the client to ensure the changes we implemented were effective, that they were resolving the problems and that we had freed up the team to do what it does best.

This story is not over yet, nor is the learning. We’re continuing to develop the product, listening to the users and making improvements. We’ve maintained the controls, the reviews, the reporting and the communication. Provided we keep the dialogue open, keep talking and supporting each other, we should continue to improve the product and the team, and to manage whatever lies ahead.