AI Agents: Who's Accountable for Mistakes? | Erich Barlow

Going further into that, you write about systems that technically achieve the goal that we're given and they actually cause harm doing it.

So let's talk about that.

Perhaps you can share some cases from financial services like bank or lender.

I'm looking for your perspective and more fintech direction, if you can share that.

Yeah, there's a lot of different things that have happened or I can share kind of thing.

Mostly when financial tech organizations or finance organizations deploy technology and It's usually around the space of helping support customer delivery of services.

And so that's where the focus with a lot of artificial intelligence is at right now.

They're looking at improving performance, cutting back on fraud.

So you'll see it within the fraud detection space, as I mentioned previously, where it's looking at holistically about withdrawals, like you withdraw money at 2.30 in the morning where you don't normally do.

that sort of thing.

Or you're looking at online transactions at weird hours or early sites that you go to.

So it's looking, it's doing pattern recognition.

Organizations run into developing and implementing those ai solutions in a manner that allows them to react to those potential threats um in a in a machine speed right so they want to be able to do that quickly and provide accurate the thing about it is is sometimes they overreact where you have uh out of your normal patterning will do something and it will say, okay, I need additional content.

So it may freeze the transaction.

I've personally seen that a lot of organizations will do that automatically as part of their approach to preventing potential fraud.

And they want to just double check, okay, did you mean to do this twice? Did you do this for a certain amount? Do you know that you did this? I know a lot of us have automations when it comes to making payments, and that also comes into play as well.

where automated payments can also trigger an AI in a way that is unexpected.

So that's kind of where organizations are deploying them, mostly fraud, mostly in areas where it can help prevent potential threats or help the customer.

You're already seeing that across the board.

Everything from communication, chatbots, those are still AI, and those are still very limited, but they definitely are employed by financial institutions as a means to help provide better services to their clients.

If we talk about just that application level, support, chatbots, how much harm that could make to the business? Not that much, probably frustrated clients, a lost business opportunity or a transaction or something like that.

Yeah, you'd be surprised.

When somebody goes onto a site and you have a chatbot, the chatbot collects quite a bit of data.

They also collect metadata about the user.

So you're collecting information about the system, you're collecting information about the browser.