DORA Metrics to Measure DevOps Performance

Look, we know the software development process is not an easy one to measure and manage, particularly as it becomes more complex and more decentralized. In many companies, there are multiple teams working on smaller parts of a big project—and these teams are spread all over the world. It’s challenging to tell who is doing what and when, where the blockers are and what kind of waste has delayed the process. Without a reliable set of data points to track across teams, it’s virtually impossible to see how each piece of the application development process puzzle fits together. DORA metrics can help shed light on how your teams are performing in DevOps. 

Well, these metrics didn’t just come out of thin air. DORA metrics are a result of six years’ worth of surveys conducted by the DORA (DevOps Research and Assessments) team, that, among other data points, specifically measure deployment frequency (DF), mean lead time for changes (MLT), mean time to recover (MTTR) and change failure rate (CFR). These metrics serve as a guide to how well the engineering teams are performing and how successful a company is at DevOps, ranging from “low performers” to “elite performers.” They help answer the question: Are we better at DevOps now than we were a year ago?

The DORA research results and data have become a standard of measurement for those people who are responsible for tracking DevOps performance in their organization. Engineering and DevOps leaders need to understand these metrics in order to manage DevOps performance and improve over time.

Simple. They help DevOps and engineering leaders measure software delivery throughput (velocity) and stability (quality). They show how development teams can deliver better software to their customers, faster. These metrics provide leaders with concrete data so they can gauge the organization’s DevOps performance—and so they can report to executives and recommend improvements.

DORA metrics help align development goals with business goals. From a product management perspective, they offer a view into how and when development teams can meet customer needs. For engineering and DevOps leaders, these metrics can help prove that DevOps implementation has a clear business value.

Let’s dig a little further into the four metrics that the DORA team has identified as being essential to an organization’s DevOps success.

Deployment frequency refers to the cadence of an organization’s successful releases to production. Teams define success differently, so deployment frequency can measure a range of things, such as how often code is deployed to production or how often it is released to end users. Regardless of what this metric measures on a team-by-team basis, elite performers aim for continuous deployment, with multiple deployments per day.

Mean lead time for changes measures how long it takes a commit to get into production. It helps engineering and DevOps leaders understand how healthy their teams’ cycle time is, and whether they would be able to handle a sudden influx of requests. Like deployment frequency, this metric provides a way to establish the pace of software delivery at an organization—its velocity.

How long does it take a team to restore service in the event of an unplanned outage or another incident? This data point is the team’s mean time to recover. It is critical to be able to restore service as quickly as possible (with a low mean time to recover). Elite performers improve this metric with the help of robust monitoring and the implementation of progressive delivery practices.

A team’s change failure rate refers to how often their changes lead to failures in production. Rollbacks, failed deployments, and incidents with quick fixes—regardless of the root cause—all count toward the change failure rate. Like the mean time to recover, this metric helps measure stability. How much developer time is diverted into tasks that don’t contribute to business value? Understanding the change failure rate helps leaders decide where to invest in infrastructure to support development teams.

At any software organization, DORA metrics are closely tied to value stream management. A value stream represents the continuous flow of value to customers, and value stream management helps an organization track and manage this flow from the ideation stage all the way through to customer delivery. With proper value stream management, the various aspects of end-to-end software development are linked and measured to make sure the full value of a product or service reaches customers efficiently.

The idea comes from lean manufacturing practices, in which every step of the physical process is monitored to ensure the greatest efficiency. In terms of software delivery, multiple teams, tools and processes must connect with each other to gain clear visibility and insight into how value flows through from end to end. This means having a platform that scales easily and enables collaboration, while reducing risk. It means accessing metrics across various development teams and stages, and it means tracking throughput and stability related to product releases. The goal of value stream management is to deliver quality software at speed that your customers want, which will drive value back to your organization.

As a proven set of DevOps benchmarks that have become industry standard, DORA metrics provide a foundation for this process. They identify points of inefficiency or waste, and you can use that information to streamline and reduce bottlenecks in your workflows. When your teams’ DORA metrics improve, the efficiency of the entire value stream improves along with them.

Is your organization better at DevOps this year than last? DORA metrics can help tell you—and using hard data, they can also demonstrate that your DevOps goals have a direct impact on the value stream. This helps unite disparate teams across your entire organization to focus on the same goals around driving business value.

Register for DevOps World and attend the Effectively Using DORA and SPACE Frameworks to Improve Value Stream Management & Delivery [1053] session to learn more!

This blog post was originally co-authored by Michael Baldani and Deepro Basu and first published on October 21, 2019. Kara Phelps has since updated it for freshness and continued relevance.

• Read more about value stream management

• Understand the research about challenges that engineering leaders face measuring team performance

• Watch the on-demand webinar: “Aligning High-Performing Engineering Teams to Business Value”