How to destroy a scammer’s business

If you’re in the scam-stopping game, it may seem odd to hear that stopping more scams is the wrong idea… but it might be.

I’m Sam, and have spent the last 4 years as an engineer at Monzo, tackling scams.

A little of a lot, or a lot of a little?

The most common type of scam is a purchase scam. Think buying something on Facebook Marketplace that never arrives. Customers report these scams in massive volume, thousands per day, which makes moving any metric around them incredibly hard.

I remember our first attempt to nudge this metric was a model aimed at all purchase scams, trained on thousands of customer reports. It was great, blocking hundreds of scam payments each week. Our “scams prevented” dashboards looked fantastic! However, the number of scams getting reported by our customers barely moved…

An interesting thing about fraud is that the majority of scams go to a minority of banks. So if you do a great job tackling scams at even one of these banks, you can have a big impact. With this in mind, later down the line we tried something different. Instead of a model across all purchase scams, we shipped a model trained only on purchase scams being paid to one bank. Again, it was great and blocked hundreds of scams each week. As with our previous broader model, the “scams prevented” dashboards looked great, we were clearly stopping more fraud. However, this time the number of scams reported by our customers fell by almost half. What changed?

Sketch of two scam spaces. All purchase scams is a large grid with a few blocked dots, labelled our model blocked a little of a lot. Purchase scams paid to one bank is a small grid inside it with most dots blocked, labelled our targeted model blocked a lot of a little

Putting scammers out of business

Imagine you’re a scammer with a long queue of victims. Each day you work through your queue trying to scam the unfortunate people who have fallen into your net. If 1 in 10 of your efforts gets blocked by the bank, it’s no big deal, you shrug it off and move on to the next. However, say 6 out of 10 efforts start getting blocked by the bank, that probably is detrimental to you. It’s not feasible to continue scamming when over half your efforts fail. Your options are to pivot, or ideally, give up.

Sketch of scammers at the end of victim queues. Spread thin: each scammer loses one in ten attempts and shrugs. Concentrated: two scammers lose six in ten attempts, are furious, and go out of business

When we shipped our purchase scam model, we were in the 1 in 10 camp, blocking a small number of efforts coming from a large number of scammers. Our one-bank model was in the 6 out of 10 camp, blocking a large number of efforts coming from a small number of scammers, and this was putting them out of business. The great thing about destroying a scammer’s business is that not only do you save the payments you block but also all the victims the scammer had in their queue.

The lesson here was that it’s better to block 50 attempts from 1 scammer than 1 attempt from 50 scammers.

The hard part

This was a great learning, but there are some real difficulties with targeted controls too. A common approach to shipping targeted controls is building them around individual scam types. The downside of this is your controls become extremely brittle. Fraudsters pivot around them the day after you ship. Building a control to stop the sale of a specific good on Facebook is a great example of this. Days after shipping, scammers start selling something else.

Nailing this approach requires finding narrower slices of payments where you can get the massive precision needed to catch the majority of scams in the space, but not so narrow that fraudsters can pivot around your controls the day after you ship. Easy to say, hard to do. There are a couple of great opportunities for this, but for obvious reasons I won’t list them on a public blog post.

If you’re working on this, reach out and we can talk.