I want to start off with a very basic question, which is just what is an eval exactly, and how does it relate to benchmarks and to RL environments?

So an eval as well as an RLM and benchmarks are just various ways to quantitatively measure the performance of an LM at a task.

And this is very important whether you're a user of a model trying to decide what model to use for a specific use case or a researcher at a lab trying to improve your models at a task.

And the way that an eval works is that it consists of multiple test cases.

And each of these test cases has three components, which is a prompt.

This is the task that the model needs to complete.

an environment that the model is situated in, and a grader that takes the work that the model completes and assigns a quantitative score to it.

Can you kind of talk about how this failure mode shows up in GBA Eval, and also just briefly describe what GBA Eval is?

What do you think about when you see someone with lung cancer these days? How's that person different?

Thank you, Bill, for your kind invitation to speak.

Even in my training, I trained in the 1980s and early 90s when we were doing big open surgeries.

Now, you've gone to minimally invasive, video-assisted, and then there's robotics now.

And even with robotics, actually, we're very proud that we're the first in the surrounding states to use what's called single port.

So we make a, like, one and a half inch incision underneath the ribs.

And we can get four little arms of the robot through that port and to do our lung cancer operation.

How often are you seeing that kind of approach?