Bridge the Business-DevOps gap with agile practices

This post was written by Matthew Skelton co-author of Team Guide to Software Operability.

Some organisations are attempting to adopt DevOps without really understanding or
adopting agile practices for software product development, and so are missing out on significant opportunities for improved innovation, delivery, and operation of their software systems. This gap between DevOps and ‘product’ or ‘the business’ is best bridged with agile practices.

How understood is agile?

I was recently chatting to the UK head of a medium-sized IT outsourcing company (3000+ people) about DevOps and in particular how agile practices are now effectively mandatory for new public sector software projects in the UK. This seemed to reflect a recent ‘pivot’ away from traditional over-planned software projects, helped by reports from Gartner, Forrester and friends that have identified agile-based software delivery as significantly more effective for most scenarios than traditional ‘waterfall’ methods.

However, are agile practices really understood and used in organisations, especially those adopting DevOps? Based on my recent experience and from talking to people in the industry, it seems that many companies assume that DevOps is merely infrastructure automation, or build & release engineering, or scripted server monitoring, and so on. Obviously, these things are valuable to do, but in themselves are really just local optimisations, leaving the delivery and innovation capability of the organisation only slightly improved.

The DevOps-Business gap

It is clear that some organisations with DevOps programmes do not really understand the value of agile practices, and so are failing to benefit from the potential huge gains to be made from taking an agile (or lean) approach to building and operating their software: an approach that really believes in experimentation, failing fast, minimal viable product, and driving decisions from empirical metrics.

In effect, there is a gap between so-called DevOps activities (automation of infrastructure, builds, deployments, configuration changes, monitoring, etc.) and how ‘the business’ sees things, which is often still through the lens of Capex/Opex budgets or pre-determined delivery deadlines, arrived at with no consultation with the technology teams, and with little or transparency across the organisation.

Some examples will highlight this DevOps-Business gap:

A while ago I did some work at a company where the head of software development would insist that the company had “very good agile practices” and yet work was routinely planned and executed using fixed-date, fixed-scope ‘projects’. Project Managers would agree scope and delivery dates with budget-holders ahead of the teams’ sprint planning sessions. In fact, Project Managers would ask Dev teams to revise their estimates until the estimates matched the pre-determined ‘story points’ available for the work!
We did some work for a successful organisation that was starting to adopt DevOps and asked us to help them with their DevOps strategy. We found that the programme sponsors (‘the business’) were of the view that limiting the work in progress would slow down delivery; instead they wanted to create additional development teams in the belief that this would make delivery quicker, even though the IT Operations team was shared by the development teams and so acted as a bottleneck.
Whilst running a training course recently for a tourism company in London, it became clear that the head of delivery was making choices that were hurting the shape of the software. Not only did he believe that Continuous Delivery was essentially ‘just’ a few Jenkins pipelines (and so wasn’t interested in core practices), but he also had chosen to manage multiple product workstreams in the same way as he’d previously managed a single workstream, with the result (thanks to Conway’s Law) that the new ‘greenfield’ system was rapidly becoming as monolithic as the ‘legacy’ systems he was hoping to replace!
When we were running a DevOps workshop in Delhi, India recently, I was (inwardly) astounded when chatting to two attendees from a large corporation when they said they had not heard of Continuous Integration (CI). For people writing software today not to be familiar with a core agile practice such as CI is baffling, but I think indicative of the absence of agile practices within large organisations for so long.

There is possibly a pattern emerging of organisations prioritising the benefits of DevOps-inspired automation over real agility. The ‘easy wins’ of DevOps are automation and standardisation of things like builds, deployments, testing, environments, and metrics collection, and organisations that do these things rightly feel the benefits. However, without a real agile approach to managing product and programme development, an organisation is crippled by deadline-driven delivery and the ever-increasing burden of technical drag that this produces.

Conversely, DevOps coupled with agile practices produces an environment where the validation of ideas by experiment is valued and understood, where new things can be tried without huge theatre and politics, and ‘the business’ can realise the vision for products and services. We have therefore begun to expand and enhance our agile advocacy at Skelton Thatcher Consulting; in fact, we’re very excited about some forthcoming changes, so “watch this space”.