Excellence is possible. We have seen more and more organization adapting effective techniques to software delivery and actually delivering projects quickly, reliably and safely.

For organizations seeking guidance on how to improve, we would like to point out that you need to start with the foundations. Once you have the foundations in place, then adopt a continuous improvement mindset by identifying your unique constraint. Once those constraints no longer hold you back, repeat the process. In this article, we are providing guidance on the 5 measures of your foundation.

Organizations increasingly rely on their ability to deliver and operate software systems to achieve their goals. To compare performance on this key outcome metric, the industry needs a way to measure the effectiveness of their development and delivery practices.

Rigorous statistical methods are used to examine how teams are developing, delivering, and operating software systems. These stats can then be used as benchmarks to understand effectiveness of a team and assess your teams performance ie are you an elite, high, medium, or low performer in the context of multiple important benchmarks.

We call these five metrics software delivery and operational (SDO) performance metrics. They provide a high-level systems view of software delivery and performance and predict an organization’s ability to achieve its goals. The five SDO metrics are:

1. Lead Time: that is the time it takes from having a PR submitted to releasing the same code in the production environment.
2. Deployment Frequency: how often the deployments are performed.
3. Change Fail: what percentage of releases to production result in degraded service and subsequently requires remediation ie rollback, hotfix etc
4. Time to Restore: How long does it generally take to restore service ie fixing a critical bug or service outage
5. Availability: availability is about ensuring a product or service is available to and can be accessed by your end users at all times.

Many professionals approach these metrics as representing a set of trade-offs, believing that increasing throughput will negatively impact the reliability of the software delivery process and the availability of services. For example, they think that by focussing on decreasing lead time, a compromise will have to made on quality or stability of the features developed. However, 6 year long research performed by DORA State of DevOps has consistently shown that speed and stability are outcomes that enable each other.

In addition to speed and stability, availability is important for operational performance. At a high level, availability represents an ability for technology teams and organizations to keep promises and assertions about the software they are operating. Notably, availability is about ensuring a product or service is available to and can be accessed by your end users. Availability reflects how well teams define their availability targets, track their current availability, and learn from any outages, making sure their feedback loops are complete.

Low performers vs elite performers

Deployment frequency

The elite group reported that it routinely deploys on-demand and performs multiple deployments per day, consistent with the last several years. By comparison, low performers reported deploying between once per month (12 per year) and once per six months (two per year), which is a decrease in performance from last year.

Change lead time

Similarly, elite performers report change lead times of less than one day, with change lead time measured as the time from code committed to having that code successfully deployed in production.

Time to restore service

The elite group reported time to restore service of less than one hour, while low performers reported between one week and one month.

Change failure rate

Elite performers reported a change failure rate between zero and 15%, while low performers reported change failure rates of 46% to 60%. The mean between these two ranges shows a 7.5% change failure rate for elite performers and 53% for low performers.

At Digital Dom we can help you make sense of these metrics, apply it to your company to continuously improve on your development and delivery capabilities. Start measuring today to improve tomorrow!