When it comes to security, fixing problems before they are exploited is easier and cheaper than doing incident response. It’s clear that fast patching stops attackers from getting in, and using best practices around cloud instances or application deployments eliminates whole swathes of issues before attackers can use them. Why is it so hard to track this work and show its value? And why is it so hard to make patch management processes stick?
For chief information security officers (CISOs) who want to prove the value they deliver, something like an individual patch is too small and too technical for company leadership to care about. However, looking at patching and remediation over time can show specific business and security problems that are definitely worth leaders’ attention. Tracking the right metrics can help your team work more effectively, but you should also be able to use this data to demonstrate your value to the business.
MTTR: What It Does (and Doesn’t) Measure
Mean time to remediate (MTTR) is the typical statistic CISOs review around patching. MTTR covers the average time it takes to get a patch into production after it is announced. It provides an overall measure of how quickly you can implement changes. However, on its own, it does not offer a huge amount of detail or demonstrate where your effort goes. It also does not show any problems that come up during patching and remediation.
One problem with MTTR is that it treats critical security vulnerabilities and minor issues equally. Consequently, some CISOs track MTTR for critical issues separately to demonstrate how they prioritize serious issues and how quickly they handle them. The other challenge is that often, one patch does not address one problem; you may have to deploy several patches, make configuration changes, and alter a registry key to call an issue “fixed.”
One CISO I know changed MTTR to “mean time to reboot” because changes may not be fully deployed (and the vulnerability handled) until after a system reboot. Some critical systems are difficult to shut down outside a specific downtime window, affecting security overall. Changing the metric’s name to “reboot” makes it clear when the team completes the patch process and ensures company leadership understands the impact.
MTTD, MTTP, MTTC: Other Metrics to Consider
Many CISOs want more detail on the process around patching and remediation. Three common metrics that show how well your processes are running are mean time to detect (MTTD), mean time to prioritize (MTTP), and mean time to communicate (MTTC).
MTTD covers how quickly your team can find and report on your current patching status, particularly when new issues are released. From an operational side, this should show how quickly your team translates any new issues released during Patch Tuesday into reports about internal issues. After detection, MTTP covers how quickly your team can prioritize issues, deciding which ones must be dealt with as critical risks and which can be fixed in time.
The sheer number of patches and updates can be daunting for security teams. However, not every update is associated with a risk. In Qualys’ “2023 TruRisk Research” report, we examined the 25,228 software vulnerability issues assigned a Common Vulnerabilities and Exposures (CVE) entry. Of these, 7,786 vulnerabilities had potential exploits, but just 159 had weaponized exploit code, and only 93 were exploited by malware. Rather than looking at thousands of potential problems, it’s important to concentrate on the biggest risks.
MTTP tracks your ability to understand the assets in your IT estate — deployed applications, services, and infrastructure — and map them against new issues and their severity. It also uses your organization’s risk management strategy to prioritize which fixes to implement first based on your deployment approach, mitigation plans, and business operations. Being able to prioritize quickly indicates your operational team is effective in translating your strategy into real-world situations.
MTTC is a new metric. It looks at how quickly the security organization can collaborate with other departments or teams involved in running IT operations or implementing updates. IT security teams may flag risks and vulnerabilities for fixing, but they may not be responsible for rolling out the patches themselves. For large enterprises with multiple teams responsible for specific areas of technology, communicating effectively can be the difference between efficient and slow deployments. Tracking MTTC can help IT security flag operational performance, but it can also demonstrate where collaboration across teams is working well and where it can be improved.
MTTC can also demonstrate potential issues in the business around risk, including teams having different priorities or teams not being responsible for specific issues. MTTC can show where these issues exist and help the entire company improve, which wouldn’t be apparent if looking only at MTTR. This can also be an opportunity to align multiple teams around incentives so that security and risk management are factored into everyone’s goals.
Show the Value of Security to the Business
Over time, tracking your success around patching and remediation can show how effective your risk management and IT security processes are. It can also be the starting point for conversations around wider attitudes about security, such as getting security involved earlier in the software supply chain and development lifecycle, and how to collaborate more effectively to make processes and workflows “secure by default.”
However, all these metrics need to be adopted across the business. The CISO and the CIO need to agree this is how they will manage the business and put it in place across all teams. They must also confront any pain caused by deploying patches faster than IT/ops teams would ideally like. Finally, they must concentrate on automating patches so that everyone can focus on risk mitigation. This is a companywide challenge, not just for the CISO. By getting the right metrics in place, you can show the value of security over time.