Is that right?

I think when we started the project, we wanted to know how language models were going to generate these epistemic markers and how they were going to express certainty and uncertainty.

And we wanted to know what the error rates were and so on and so forth.

But before we could even answer that question, we first had to deal with the idea of do language models even understand what these markers even mean? Do they even understand what epistemic markers are? And so we did a much easier experiment to start off, which was that first paper in the series of three papers we wrote, which was if I prompt a language model with saying, What is the capital of France? I'm certain the answer is blank.

And we have the language model fill in the blank.

We wanted to know, could it recognize that if it says, I'm certain the answer is blank versus I'm not sure the answer is blank, that when it says, I'm certain it's more likely to produce the correct answer.

And when we lead it with, I think the answer is, it's more likely to produce the incorrect answer.

So what does that look like for you? Yeah.