If you ask any IT department what they measure when they run a phishing simulation, they’ll almost always say click rate. They want to know what percentage of users “fell” for the fake attack. Click rate is such a common measure for phishing simulations that some of the biggest awareness training and attack simulation platforms tout a reduced click rate as their primary objective.
This doesn’t make much sense though, as click rate is a lousy metric for phishing simulations. To understand why, it’s important to dig into the reasons for running simulated phishing attacks in the first place.
Phishing as a Test
One common justification for running phishing simulations is that it is a test of an organization’s risk exposure. It tells you how vulnerable your organization is to social engineering attacks. In this view, a low click rate would be good.
However, sending out a single phishing email to “test” your organization doesn’t measure much of anything. If you get a 5% click rate, it doesn’t mean the other 95% of users are safe. The reality is that most people ignore most emails most of the time. There are a million reasons to not click on a phishing email that have nothing to do with cybersecurity. Was the email relevant and interesting? Did the person even read it? Did they care? A one-off click rate is basically meaningless.
You might think you could get around this by sending out multiple phishing emails over time, but the problem there is that the attacks are not standardized. If you get a 4% click rate on the first email and a 3% click rate on the second, is it because your organization got better, or because your email was less click-worthy? You won’t know.
Overall, running phishing simulations to measure your organization’s performance is not a fruitful exercise, and using click rate in that context is a quantifiable figure that disguises how little we really know.
Phishing as Training
The second common justification for running phishing simulations is that it is good awareness training. It teaches people how to identify and avoid phishing emails.[1] This is the view that we hold at Pistachio.
If the goal is training, then you want as many people to fail as possible. In other words, you want a high click rate. Why? Because the people who fail phishing simulations are the least likely to fail in the future.
We can see this in our data. Users who “fail” an attack simulation are 50% less likely to fail the next phishing simulation they receive relative to the users who ignored the previous attack.[2] Falling for a simulation, and receiving the immediate feedback Pistachio provides, really does help people to become safer. So we want people to fail![3]
It’s no different than going to the gym, really. If you are always succeeding on all of your lifts then you’re never “pushing to failure” and you won’t get the results you want.[4] You have to fail to improve.
A good phishing simulation program solution should aim to achieve a high percentage of users failing at least one attack, and also should ensure that users are exposed to a variety of different types of attacks at different difficulty levels.[5] Click rate doesn’t capture that information, so it’s not a good metric.[6]
Focus on What Matters
Our aim at Pistachio is to provide your employees the training and experience they need to stay safe in the digital world. Sometimes we need to send out very challenging phishing emails to expose people to the latest techniques, and other times we need to send out emails that are obviously wrong because some people click those too. What each user needs is different, and Pistachio has to be adaptive to that.
Focusing on click rate would get in the way of that. It does not capture the objective of running phishing simulations. Next time someone asks to see the click rate on your phishing simulations, send them a link to this blog post instead.
