Ziyad Basheer on Governing AI Agents for Reliable Deployment at Aide

Thank you very much.

I'm excited to be here.

I very much enjoy chatting about agents, governance, and making sure how to run them safely in production.

So let's go ahead and get started.

So today's session, today's talk is gonna be about governed agentic AI, running agents while making sure that they're compliant, that they follow your resolution protocols, and that they adhere to all of the things that you outlined.

Now, why is this a problem, and why is governance a thing in the first place? The reason for that is when it comes to AI models and large language models broadly, they are probabilistic by nature, so they work based on next token prediction, and that introduces variance.

So if it's a probabilistic model, it's basically going off of what is the next most likely token based on the tokens that it's seen so far, and that could be problematic.

And so essentially, AI agent governance ensures that these models are more deterministic than they are probabilistic because in environments such as finance, commerce, where the stakes are high, regulation is involved, and compliance is important, we need to make sure that these AI agents are acting within the bounds of what they're supposed to be doing.

And are not creating novel behavior or are diverging from their configuration.

This is very critical when you are having conversations in contexts that are customer-facing or involve financial matters where you cannot make a false claim about someone's account balance, or you cannot make false promises that could erode your brand's trust and authority.

And so this is why we need to govern these AI agents.

Now, the next question is, now that we understand why governance is important, the next question is: how do we make it happen? Well, the very first thing is getting rid of hallucination.

So we cannot have models basically in AI agents running production that are gonna be making stuff up.

That is the biggest risk, and that is the fundamental problem with having probabilistic models.

And so for us, that is the main thing that we focused on from day one because the power of this technology is great, but how do we, as practitioners, give businesses the confidence that they need to run them in production without any risks? And so the very first step to addressing that was getting rid of hallucination.

Now, hallucination can be a feature, and in the sense that if a model does not have certain data, it is, it has the tendency to make that data up.

So this can show you what gaps exist in the data that it's pulling from.

So one of the basic ways that this can be addressed is by grounding it in real data, making sure it actually has real data to pull from.

And if you identify any information that it's claiming or using, then that is an opportunity for you to find the real data and make sure that it's available factually for the model, for the agent to be using.

The second thing is all of these language models, because they are generative, they also have the tendency to just want to create tokens, especially as they improve and model providers charge per token, so the incentive is for more tokens to be produced.

This creates an eagerness to respond for the agent, and it also make it kind of like a sycophant.

It's fascinating how identifying gaps in the data can actually turn a potential problem into an opportunity for improvement.

This kind of proactive approach is really what sets successful tech companies apart.

Where it kinda wants to please you and tell you what you want to hear.

And in any production context, that is not something that is acceptable because they need to be factual, and they need to be accurate.