By Ashu Chatterji, CEO at Caravel Labs

We know what high-performing software delivery and operations looks like, but do we dare to confront the simple truth about it?

Anyone who is in the business of developing and operating software for business use, “does DevOps”. The term DevOps itself is a portmanteau for development and operations, after all. The real question to ask is how well a software engineering performs on development and operations and receive the answer in simple terms – throughput (measured in terms of deployment frequency and lead time to change) and stability (change failure rate and median time to recover). This, by the way, is not just our own opinion, but the standard set by the widely respected DevOps Research and Assessment (DORA) team at Google Cloud who publish their annual report, which is available to all for free, and should be off immense interest to anyone depending on software to deliver value to their constituents.  

DORA publishes an annual State of DevOps report which gives a clear picture about how the highest performing software delivery organizations differentiate themselves from the rest. The latest report, which was published in 2021, shows the following: 

These metrics are simple, straightforward, unequivocal and relatively easy to measure. Most developer platforms such as Azure DevOps and Jira provide for measuring these. Software engineering teams improve their performance immensely and rapidly from monitoring their metrics and looking for ways to improve them. The reason is best described in the 2021 report itself:  

… these metrics focus on system-level outcomes, which helps avoid the common pitfalls of software metrics, such as pitting functions against each other and making local optimizations at the cost of overall outcomes. 

Anyone sponsoring a software engineering team would benefit from asking this question of their team’s leadership. If a straightforward answer is not forthcoming, it is probably because there are roles and functions within their software engineering machinery that benefit from waste. We often wonder how so many customers who have outsourced software development to vendors first pay for the skilled developers whose time they buy, and then pay for oversight and governance in the form of extra project management, technical oversight and quality control “taxes.”  

At Caravel Labs, perhaps because we believe that our customers pay us to equip them with a software delivery and operations capability, and not just to build and operate a piece of software, we have chosen to deal with the truth of our software delivery performance metrics. And we do it by daring to be fully transparent about it within the team and with our customers. Actually, not only are we fully transparent about it with our customers, but we also go out of the way to educate them about it and remind them to hold us accountable.  

The graphic below is a screenshot of the DevOps performance metrics for one of our project teams. The team, which was only a couple of months old, was deploying consistently to production on a weekly basis. In the middle of January, they committed to deploying on a daily basis in their push towards attaining Elite status. This initially resulted in an uptick in the change failure rate in the first week, as weaknesses in the quality assurance and deployment processes were exposed. Instead of bringing in manual oversight and governance roles, such as more testers and project managers, the team used their learnings to improve automated testing, make the CI/CD pipelines more robust and the infrastructure as code more efficient. A week later, the team is consistently deploying changes to production at least once a day. The team is delivering the same velocity with one less developer and the number of dedicated project managers, testers and architects on the project remains unchanged. Zero. 

At Caravel Labs, we serve customers who are changing the world for the better, whether they are social entrepreneurs or change makers in the public and private sector, or even IT companies like us who build software for such innovators. The events of the last couple of years are a poignant and somber reminder of what scientists and other experts have been saying for a while, that the window for making the necessary changes to towards a sustainable future for our planet and for our species is closing rapidly. We know that digital transformation is the key to maximizing impact and meeting this shrinking opportunity, which is why we cannot squander precious time by being low performers in software delivery and operations. We must dare to deal with the truth of our software engineering performance so that we may improve it.  

If you are a leader in a software engineering organization and are wondering how your organization is performing and how it might improve, we are here to serve. Please drop us an email at info@caravellabs.com