The Best Fraud Defences Prevent Zero Fraud

As a fraud team, it seems logical that your measure of success is the amount of fraud you prevent. You might even boast “we prevented over £1m of fraud last month 🎉”. In my opinion, this is not something to celebrate.

In this post I’ll share my take on why you should in fact be aiming to prevent as little fraud as possible, and what the real measure of success for fraud teams should be.

I’m Rob. I spent five years as a Product Manager at Monzo, building systems to fight financial crime.

Scammers are professionals. They spend their time constantly testing your defences. Different scripts, different types of payments, different times of day and so on. As soon as they stumble across a new tactic that seems to work, they go all in, and your fraud losses start growing exponentially, until the scammers are only limited by the time they have to scam.

Sketch of a scam trend over 30 days as red bars of £ lost per day. In the discovery phase scammers test your defences and losses are tiny. In the exploit the gap phase it works, so they all pile in and losses climb fast. In the scammer max capacity phase losses level off at about £50,000 a day, limited by how much time scammers have.

When you see a concentrated spike like this, we call it a “trend”: a cluster of scam reports that all look exactly the same. For example, a dozen reports, each with four card payments, all on Apple Pay, all for £250, all to the same merchant, all at 2am.

You respond to trends with a “detector”, which is either:

A rule - Block all payments to this merchant, where it’s the user’s first crypto payment, within 3 hours of an auth payment, between £100-£150.
A model - Train a machine learning model to learn the patterns of the trend, and block all payments where the model thinks there’s a high probability of it being a scam.

Your goal as a fraud team is to identify and respond to these trends as early as possible, and shut them down before they can reach the “exploit the gap” phase. Your true measure of success is not how much fraud your detector prevents, it’s how quickly you were able to deploy the new detector.

This becomes really clear with a worked example.

Example 1: You react quickly to a new trend

Let’s say you spot a new scam trend, you do a great job, and within 10 days from the first few scams appearing, you deploy a new detector that prevents 60% of the scams by value (60% £ recall).

On day 10, fraudsters are still early in the “exploit the gap” phase and might be stealing a few thousand pounds per day.

With the detector live and foiling 60% of the scam attempts, the scammers will start to give up on this new tactic. Each day fewer and fewer scams are attempted, until your detector is preventing £0 per day, since there is nothing left to prevent.

Sketch of the same trend with a detector deployed on day 10. Red bars of £ lost per day rise to about £17,000, then from day 10 green bars of £ prevented per day stack on top and both shrink away to nothing by around day 20.

Example 2: You react slowly to a new trend

Now imagine the exact same trend, but this time you react more slowly and deploy your detector on day 20.

It has the same 60% £ recall as example 1, only this time the scammers are already in the “max capacity” phase. The trend has grown so much by this point scammers are stealing tens of thousands every day!

Still, from day 20 you start preventing most of the scams, and again the losses start to tail off as scammers give up until eventually the loss per day drops to zero.

Sketch of the same trend with a detector deployed on day 20. Red bars of £ lost per day climb to about £50,000 and stay there, then from day 20 much larger green bars of £ prevented per day stack on top before both tail off.

Reacting fast vs slow: the final tally

We can calculate the total fraud loss by adding up all the red bars in those charts. Clearly, reacting slower is much worse. Reacting in 20 days means you lose a total of £495,800. React in 10 days and you lose just £54,500. An almost 10x reduction by reacting 10 days faster!

What’s crazy, though, is what you find when you add up the green bars to see how much your detector prevented. In example 2, where you react slowly, you will actually “prevent” three times as much fraud, £99,900 against £33,500! The reality is, you prevented more because you let the trend grow in the first place.

Think about the extreme case. If you had perfect, impenetrable scam defences, no scammer would ever even attempt to scam your customers as it would be a waste of time, and your total £ prevented would always be £0.

	Example 1 Reacting fast	Example 2 Reacting slow
Reaction time How long did it take from the first scam of this trend to the detector being deployed?	10 days 🏅 Winner	20 days
£ reported from trend Add up all the £ reported from day 0 until the scammers gave up on this trend.	£54,500 🏅 Winner	£495,800
£ prevented by new detector Add up the amount prevented by the new detector on each day.	£33,500	£99,900 🏅 Winner?!

How to react fast

You only get good at what you measure and practise. At Monzo, we set ourselves goals for how quickly we could spot new trends, write new detectors and get them live. This forces you to focus on the things that really matter, like data quality, dashboards, internal processes and tooling.

Each of these topics deserves its own blog post. Subscribe below so you see when they drop!