8 DevOps Metrics to Measure Team Activity & Progress

3. Deployment Frequency

Deployment frequency relates to the ultimate goal of DevOps: fast creation of high-quality software. Naturally, measuring how often the team pushes new code into production shows how the pipeline is performing toward this objective. Deployment frequency provides this insight.

However, it's important to weigh this metric against the change failure rate discussed above. If your team is deploying code frequently but that code often contains bugs, then the process is not as efficient as the deployment frequency alone would suggest. Flawed deployments will also lead to lower customer satisfaction — especially if the issues impact user experience.

4. Mean Time to Recovery

Mean time to recovery (MTTR) in the DevOps model is a measure of how long it takes for a deployed application to recover from failure and return to normal operations. MTTR does not distinguish whether the service interruption results from a code deployment or a larger system failure.

An application monitoring strategy is essential for decreasing MTTR. Monitoring and logging tools will speed time to detection and may also identify the cause of the outage. One of DevOps' strengths is the collaboration between the development and operations teams, which allows for faster identification of the issue and remediation.

5. Customer Ticket Volume

Though not selected by DORA, customer ticket volume is a critical metric for measuring how well your software is meeting the needs of your end users. Feedback is essential to the DevOps model's emphasis on constant improvement, and customer tickets are one data source to focus your efforts on addressing recurring issues.

However, a high volume of customer tickets suggests that your application is not meeting your customer's central needs. Being able to quickly catalog tickets will allow you to gain a high-level picture of issues and their severity. In any scenario, lower numbers of customer issues show that your application is meeting expectations and performing as needed. Higher volumes indicate that you need to step back and reevaluate your approach.

6. Defect Escape Rate

Defect escape rate examines the rate at which code containing bugs or other flaws is pushed into production. Like change failure rate, it is another measure of the testing process's effectiveness and the quality assurance (QA) strategy overall.

However, defect escape rate exclusively focuses on the percentage of production pushes with bad code versus change failure rate that records how many of these pushes require immediate remediation. Taken together, these metrics provide greater insight into areas where the QA process can improve to catch errors earlier in the pipeline and the impact of uncaught bugs.

7. Application Performance

Application performance inspects how the application stands up to different resource demands and meets users' needs during normal and peak operating parameters. Ideally, the application has a robust foundation to respond to large requests without impacting load times for users.

Application performance will measure how the application responds to high-demand scenarios. These tests are conducted prior to deployment to confirm the software will meet defined customer requirements. After deployment, DevOps teams will continue to monitor performance to ensure the application operates as expected in production. Decreases in application performance flag potential issues to proactively assess and correct.

8. Mean Time to Detection

Mean time to detection (MTTD) is a measure of how long it takes to identify and flag an issue once it appears in production. This directly correlates to application performance monitoring as a primary way to spot problems, though customer tickets are another source of feedback.

The sooner an error is detected, the sooner the team can investigate and introduce a fix. A short MTTD shows that your team has an effective monitoring strategy and is responsive to issues as they occur. When combined with a short MTTR, your team identifies and solves problems at scale — ideally before the customer has even noticed an issue with the application.

DevOps Metrics Dashboards

You may find additional metrics that are valuable to your team's unique DevOps model beyond the eight we've covered here. So how can we organize this information into a digestible format with graphs and visual aids instead of trying to sort through spreadsheets?

In the example from earlier, the definition of success is clear and broken down into actionable steps. Progress is reflected in the course dashboard by showing the completion bar and finished lessons.

DevOps teams can take advantage of dashboards to chart their progress and better understand complicated data.

Image Source

Dashboards are a great way to visualize complicated statistics and share them with a broader audience. Providing dashboards to your team allows them to participate in tracking goals and will inspire a greater sense of accountability. You can also use dashboards to show company leadership that your DevOps model is delivering on their goals.

We'll go over three different types of dashboards you can implement below.

1. Agile Project Management Dashboards

Image Source

Agile project management dashboards focus on measuring and visualizing the workflows specific to moving the application forward. These dashboards will show how the team is performing against goals like deadlines and task completion. They also provide better insight for setting expectations, such as the average turnaround time for creating a particular software component.

2. Application Monitoring Dashboards

Image Source

Application monitoring dashboards are dedicated to recording the performance of the application in production. This data helps establish benchmarks for normal operations, which then helps flag below average performance for further inspection.

Application monitoring dashboards can also aggregate customer tickets for patterns, record type and frequency of errors, and record MTTD and MTTR to provide a big picture view of the health of the application.

3. Platform Observation Dashboards

Image Source

Platform observation dashboards provide another view of the application's health, but these dashboards focus on the entire technology stack. This scope includes the application's containers, network, and storage to measure if the underlying infrastructure is supporting the software's performance.

Insights from these dashboards help the operations team better configure the environment to enable fast server responses to customer requests and other desired behavior.

Measure twice, cut once.

I know, DevOps isn't the same as lumber, but hear me out. The adage "measure twice, cut once" alludes to a deeper approach to data: the more information we collect, the better informed our decisions will be. In practice, this can be as simple as measuring a second time to make sure you marked the correct length on your 2x4 before cutting it.

Obviously, the DevOps model is exponentially more complex than using a saw. But like our theoretical carpenter, your goal is to gather as much data from as many sources as possible to make decisions that account for the full context of your team and pipeline. By prioritizing the DevOps metrics we've covered in this post, you'll be able to reinforce your judgment calls with hard numbers and set measurable goals to drive your team forward.