Aditya Ganjam on Conviva's User Experience Innovations

I'm very excited to be here, um, and talk to you about, uh, how we've been looking at AI agents and how to, how to measure, evaluate, and improve them.

Um, I...

And, and the key challenge we've, uh, we've seen working with our companies is that, um, uh, a-a-agent builder teams, um, spend a lot of time building evals and testing, testing agents, but still, when they go live, there are a lot of cases it doesn't work for the users.

So question, the challenge of the industry is: how do we actually capture the right signals? How do we improve, uh, AI agents? Now, before I go into that, I'll share a little bit of background about myself and, and the company, uh, uh, where I'm at, uh, Conviva.

Um, so we've been, uh, Conviva's been around twenty years.

I'm one of the co-founders.

We started it, uh, in the early days of streaming video.

Um, if you all think back maybe fifteen, twenty years when it was just starting, the quality wasn't that great.

Uh, but now, um, we have amazing quality video we can watch on any device, any time, any content we want.

Um, but that was just a vision twenty years ago.

And, uh, when we started the company, we really looked at, uh, what we felt was a core challenge at that time, which is: how do we ensure a great experience, um, we called the quality of experience, uh, that viewers would have when they tune into a streaming video? So we built, we built a solution for this, uh, with the core philosophy that to optimize that experience, we have to measure the experience from a, from the user per-per-perspective.

So what is the perceived experience that viewers have when they're actually streaming video? Um, we built a platform for this, uh, we've been working for the last twenty years with a lot of the major streaming companies, many of the streaming apps, uh, that you would all use and have installed, and helping them optimize that and continue to do that today.

For example, we just worked with about, uh, over fifteen different companies worldwide just streaming the World Cup in the last month, um, to many tens and hundreds of millions of views-viewers.

We've then expanded that into digital products, into measuring experience and optimizing experience for digital products across industries, um, B2C and B2B.

Um, and then most recently, we've been looking at the challenge of agentic experiences.

Um, as companies deploy AI agents, um, that are directly interfacing with, with, uh, consumers, how do we make sure that the consumers are having a good experience, are actually achieving what they came to the agent to achieve, um, and are not, you know, left hanging in, in between? Um, so that's what I'm gonna focus, uh, most of this, uh, the rest of this time on.

Um, I'll use one example, um, from the financial s- uh, services industry to kind of showcase what, um, what kind of interaction may exist and what kind of challenges, um, may exist.

So here's a sample conversation.

Um, a, a user goes to a bank website, um, says, "I see a few charges on my card that, um, I didn't make.

Um, so looks like they're, they're fraudulent." Um, agent is responding well, apologizes for having this issue.

Um, it confirms which ones they were and on what date.

This highlights a critical shift.

As AI becomes more integrated, ensuring a seamless user experience through agents is becoming a must-have for companies, not just a nice-to-have.

And then says, "Okay, I'll take care of this.

I'll file the dispute and, um, you'll see that on your account shortly." And the user, of course, appreciative of this.