AI Accountability in Action

Right.

And I don't want to get into this piece of litigation, but as I segue into our next topic, I'm going to mention two pieces of litigation that I'm aware of.

One was an airline in Canada that where there was an individual who was communicating with a chatbot that the airline had, and he was communicating about a bereavement fare and how to get the bereavement fare.

Long story short, the chatbot gave him the wrong answer based upon the airline's policies, and the gentleman sued and was successful.

It wasn't a huge amount, but the chatbot was the agent, as you kind of described.

I mean, it's artificial intelligence, but it was an agent of the airline.

And the court was like, well, if you had a human who was sitting back there typing... then you would have been stuck with that decision based upon what the agent said.

The other one is about UnitedHealthcare is being sued currently, and there's ongoing litigation about a denial process.

It was for Medicare Advantage plans, and they were using artificial intelligence to review claims and make or deny claims.

The allegation, because it's ongoing litigation and none of this is proven, but the allegation is that 9 out of 10 claims that were appealed and then reviewed by a human reviewer were reversed.

When we talk about this, the reality is that Less than 0.2% of denied claims were actually appealed.

So only two out of 1,000 denied claims actually were appealed and then reviewed by humans, which raises the question, what about the other 998 claims that were denied? Of course, there are probably some in there that were rightfully declined.

but how many others were wrongfully declined.

And as we kind of jump into this next topic, the challenges that the courts, regulators, insurers, businesses, we're all kind of wrestling with is the telemetry and the explainability of the AI models decision-making process.

How do we as users justify, how do we trust these products? How will we hold them accountable? And how do we look at being able to reconstruct the decision-making process to justify whether the decision was right or wrong in the first place?

So you bring up a great point.

I work with a lot of regulated industries and I also work within a lot of the startup and innovators.

This is an area that's really changing because you have to be able to give that explainability.

So you're seeing the models and the technology around the models is giving more of that we'll call transactional receipts.

You're also finding where people can guide the model to making better decisions.

That's important.

The other piece, and this is something a bit of a new transition, I can get business insurance to cover a lot of things.

I can get coverage that if an agent or an executive makes a mistake, I can have coverage for that.

Up until recently, they didn't exist for AI.

So we talked about a little bit earlier, I'm starting to see a trend where companies that provide some kind of AI augmented services, they're getting their legal teams, their privacy, their security or operational business teams.