Why is it so Difficult to Scale Agile Software Development?

October 2, 2020

Scaling Agile software development is no easy task. But obtaining agility in managing change can make or break an organization, and those who are successful will win the race for the talent of the future. Organizations that can grow without losing the ability to adapt to continuous change are those that manage to clearly define their goals and expectations toward their employees. If you’re struggling to manage change and scale Agile software development, here are some great tips to help.

20-year-old Agile Manifesto

Agile software development has gained a large foothold in the IT industry. At this point, it’s an the exception to the rule if you find a job advert for a software developer or project manager responsible for digital deliveries where the word ‘agile’ can’t be found somewhere.

The ability to structure your organization so that it can adapt to changing market needs is a valuable competence for any leader. Here, agile development fits in well because the methods and processes are designed not toward the final product, but toward an iterative process where product development and quality improvements are continuously evaluated and assessed before the next iteration begins.

The problem is that what works well in some contexts does not necessarily work well if you scale up your organization.

The processes and agreements that work well for three interdependent teams don’t necessarily work for 30 interdependent teams.

The processes and agreements that work well for three interdependent teams don’t necessarily work for 30 interdependent teams.

Why is it Difficult to Scale Agile Software Development?

One of the pitfalls of scaling agile software development is that when you take what works in a small organization and try to replicate it in a large organization, it can cause major problems. Without taking the increased complexity and required change management into account, scaling agile can be troublesome to replicate when hundreds or thousands of knowledge workers have to work according to the principles of an agile manifesto that was formulated with smaller, independent, competent units in mind.

The industry has tried to remedy this by inventing new processes and tools to scale agile software development so that a supply organization of tens of thousands of employees can work agilely.

The framework that currently has the largest market share is SAFe. Although, there are other, perfectly good alternatives, where the value set behind differentiates; LeSS is worth mentioning, as I know its the only one addressing the real problem right from the start: Context means everything when you need to find the right solution to a given problem.

it is easy to lose yourself in mastering a new process marketed as if the process itself is the cause of the problems you have experienced if you haven’t established metrics focusing on the end-user experience or if you don’t have an opinion of how quality should be measured in the software you provide.

Tips on Working Together to Scale Software Development

The agile manifesto was written 20 years ago as a reaction to rigid waterfall processes, where there is no willingness to take risks. It was warranted as management didn’t understand that if you manage your risks properly, changing course later on won’t necessarily be hugely expensive.

If you work in the physical world, faulty calculations can be a costly affair.

For example, once you have poured the foundation for a house, it will be a major undertaking to enlarge the ground plan if you wish for your house to be 200 square meters instead of 180 square meters.

Such is the case in the physical world where we can’t control the laws of physics. But in software development, it is different.

Thus, as a manager, you can afford to manage risks in a completely different way and build a delivery device that works fundamentally differently from traditional project models—but only if you understand the differences and are able to shape your organization accordingly.

It is good that the top management has understood that it must deal with digitization and how software is created in the organization.

Understanding is one thing, knowledge of causes and effects is another story.

If you want to more value for your money, it won’t come just by introducing a new software development process such as SAFe or LeSS. However, it is easy to lose yourself in mastering a new software process marketed as if the process itself is the cause of the problems you have experienced. This is especially true if you haven’t established metrics focusing on the end-user experience or if you don’t have an opinion of how software quality should be measured.

Prior Approval is not a Prerequisite

 It’s also not a prerequisite for quality in a large delivery organization that a release from a team must be approved several days in advance by people who aren’t involved in the development and who won’t be sanctioned if the release is erroneous.

On the other hand, it’s essential to be able to integrate deliveries from different teams early and often to be able to continuously assess whether they can function as a whole. Here, it’s your job as a leader to understand why large-scale test automation is a prerequisite for a speedy software development process.

There is a low, upper limit to both quality and workflow if one relies on manual tests to give developers feedback on the quality of their deliveries.

What does the number of hours spent on story points say about the quality of the end product? Nothing—absolutely nothing.

Estimates are interesting for the individual teams as input to their constant improvement processes. However, the number of hours spent on a delivery does not correlate with the users’ experience of quality software in the end product.

Descale the Complexity

To successfully scale Agile software development, it’s not about increasing complexity.

On the contrary, the criteria for success is instead to descale complexity and structure the organization around a technical platform that ensures that teams can assess the quality of their own deliveries early and often.

This demands that you adopt another form of risk management than traditional waterfall project management, ensure full transparency in the portfolio management of initiatives and roadmaps, allow flexibility in how a delivery is made and that you as an organization set common goals for the end-user quality.

From here, you shouldn’t care what your process is called. Context is paramount, and there is no one-size-fits-all process. Although, it’s easy to fall in love with process tools that, on the surface, promise to solve a variety of problems but do not skilled knowledge workers are prevented by performing optimally due to the root causes of low-speed problems: opaque tasks progression and low-quality end products are embedded in your organization in terms of culture, organizational complexity and technical debt.

On the contrary, the criteria for success is instead to descale complexity and structure the organization around a technical platform that ensures that teams can assess the quality of their own deliveries early and often.

Your Role as an Agile Leader

As a leader, you need to set the direction and expectations for what, when and in what quality software should be delivered.

You as a leader don’t have to be deeply engaged in how a team develops software, as long as it delivers on time, roughly within budget and lives up to the quality expectations that you have set on behalf of the organization.

If a team executes a sprint solely using post-it notes on the wall and is happy with it – well, let it do so as long as the portfolio management of upcoming tasks and initiatives is done in a tool that is common to your entire organization.

If it uses Daily Scrum—fine. If it doesn’t or only does it every three days, it’s probably fine too, as long as other teams do not feel they lack information or access to the team.

If a team wants to run iterations with a four weeks duration when the rest of the organization runs in sprints of two weeks, then it’s interesting to learn the reasons for this. Yet, the team shouldn’t be discouraged if it makes sense in context and doesn’t cause friction or bottlenecks elsewhere in the value chain.

Call it agile development, waterfalls, DevOps, SAFe, LeSS or something else, it doesn’t matter.

The organizations that will win the race for the talent of the future and who can grow without losing the ability to adapt to continuous change are those that manage to clearly define their goals and expectations towards their employees.

From there, the upcoming winners will dare to let talented employees find the right path by bringing their skills and experience into play—for the joy and benefit of both their employer and not least the end-users and customers who, ultimately, are the main stakeholders in any business.

 

Further Reading

Here is relevant material in the form of books on Amazon that can provide inspiration on how to descale complexity and implement change without getting lost in the process along the way.

 

  Implementing Lean Software Development (Mary and Tom Poppendieck)

 Leading Change (John P. Kotter)

 The Goal – Theory of Constraints (Eliyaho M. Goldratt and Jeff Cox)

DevOps Engineer, Agile Coach

DevOps consultant, Agile coach and full-stack software developer with 15 years of experience in the software industry.

Passion for holistic thinking, automation and implementing data-driven decision processes.

Core competencies

DevOps and Agile practices
Scalability
Security
Software development
Software architecture

Past experience
Energinet, LEGO, DGI, Just-Eat

We can Help!

We provide consultancy on how to create great digital products.

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Learn more about our consulting services

Get in touch with us