Key metrics to measure the success of a Red Team Exercise#

Red team exercises are an important part of an organization’s security strategy, but it’s important to measure the success of these exercises in order to determine their effectiveness. There are a number of different metrics that can be used to measure the success of a red team exercise, including the number of successful attacks carried out by the red team, the number of vulnerabilities identified, and the time it takes the blue team to respond to the attacks. Measuring the success of a red team exercise can help organizations determine whether they need to make changes to their security strategy and can help improve the effectiveness of these exercises.

What to measure#

Here are three key metrics to measure the success of a red team exercise:

  1. Effectiveness of attack - One of the most important things to measure is how effective the red team’s attacks are. This includes things like the number of systems compromised, the amount of data stolen, and the duration of the attack.

  2. Time to detect and respond to the attack - Another important metric is how quickly the organization detects the attack. This includes things like how long it takes for security teams to notice unusual activity, how long it takes to identify the attack, and how long it takes to contain the attack.

  3. Lessons learned - Finally, it’s important to measure how well the organization learned from the exercise. This includes things like identifying how the attackers were able to penetrate the defenses, what could be done to improve detection and response times, and how to reduce the impact of future attacks.

Here’s a non-exhaustive list of items to evaluate when reviewing the results of a Red Team exercise:

Operational Goals#

  • Did the defender detect the C2 channel?

  • Could the defender identify all of the Red Team’s external C2 infrastructure?

  • Could the defender remove the Red Team from the network and prevent them from coming back?

  • Did the defender convert their incident response artefacts into cyber threat intelligence that can prevent future attacks and be shared with 3rd parties?

  • Which procedure(s) helped the Blue Team respond to the Red Team vs. which ones did not?

Other Goals#

  • Which security products detect the Red Team vs. which ones did not? Why not?

  • How much interest did the business executives show in the results of the Red Team? Did they care? If not, why not?

  • Did we miss some key skills and capabilities when trying to stop the Red Team?

The risk of not measuring#

Without understanding the business value, it’s often difficult to justify the investment in a Red Team exercise.

Measurement and evaluation of a red team’s impact on the organization is critical to ensure that the team is providing value and that their efforts are having the desired impact.

Organizations should take the time to identify which metrics are most important to them and then track these metrics over time to evaluate the effectiveness of their investment. By doing so, organizations can ensure that their red team is having a positive impact on their security posture.

Conclusion#

A successful red team exercise is one that meets the objectives set out at the beginning. By measuring the key performance indicators, you can ensure that your red team exercise is successful.

See also

Looking to expand your knowledge of red teaming? Check out our online course, MRT - Certified Red Teamer In this course, you’ll learn about the different aspects of red teaming and how to put them into practice.