80: “Dark Side of the Boom” Author Patrick Coughlin on AI Scams

Okay, let's back up.

So who was it? Why do you think they were targeted? 'Cause to your point, you know, a nice, lovely lady in Kansas, she would not be the highest priority, right?

That's right.

She is nice and lovely.

She [laughs] but look, I think what's happened is AI in the hands of cyber criminals has made it cheaper, easier, faster, more scalable to target more people with more personalized content than ever before.

These kinds of attacks weren't profitable to be pointed at consumers in the past because they simply took too much work.

I mean, if you think about it, that attacker had to know my mom's phone number, roughly her address, 'cause they referenced a Walmart that was near her.

They had to know my sister's phone number, my sister's address.

They had to, you know, have some idea about what my mom's living situation was.

They knew she lived alone, et cetera.

So somebody did some work here, and then maybe they cloned my sister's voice.

We were never able to confirm that or not, but we found some social media posts that showed us it would've been possible.

So a ton of work is done here, and that takes time, energy, and effort.

And the ask, I didn't even tell you what the ask was.

They asked for $1,200 in exchange for saving my sister.

And so what that tells you is somebody would've been ringing a gong in, like, a, a scam syndicate in Southeast Asia and high-fiving if they had gotten $1,200.

That was the ask.

That was the big score, and so you know it must have cost a fraction, a tiny fraction of that to actually execute that, and that's what AI has done.

It has allowed at every stage of the scam, it has increased the automation and the precision of the convincing content that is used to get everyday people to become victims of these types of scams.

scams.And so it came from overseas.

You traced it back.

It was impossible for us to trace it with what we had at the time, and we could talk more about why this is so hard and some of the regulatory and incentive gaps in these platforms to actually enforce laws and what makes it really a challenge.

But ultimately, you know, we believe it sort of had the trappings of an overseas actor.

These types of scams originate from really all kinds of places, Southeast Asia, they come from Eastern Europe, they come from Western Africa, and they come from closer to home too.

There's plenty of homegrown organizations that do this as well.