Arena.ai

I will just want to repeat this directly to the camera.

Folks, Peter Grosto from Arena AI is, I think, summarizing the vibes of the whole timeline.

It does not make sense to use Fable or Asteroid now.

This is where we are.

Gemini for Argon, Google's new frontier models for trusted testers only.

We have such a trusted tester here with us.

Have you and Arena played with this model? What do you

think? We have Arena's coin.

By caskie.arena.ai measured how Claude Opus five point five writes against Opus five, the version before it, and a large part of the change sits in two punctuation marks it calls familiar tells of AI writing.

Opus five point five has nearly stopped using one of them.

Downward trend, how Arena measured it.

The comparison uses high reasoning answers from Text Arena.

BleepingComputer, which reported on the analysis, dates those answers to August and September twenty twenty-six.

Arena tracked twelve writing measures.

Ten moved in what it calls a better direction.

Neither the sample size nor the full method has been published, so the results describe Arena's set of answers and make no claim about all Opus five point five output.

Those two numbers, right? They set the ceiling for every infrastructure deal after them.

Second, Claude Sonnet 5.5 goes live on Arena's Direct Mode starting today.

Run your own head-to-head against GPT 6.1 Sol once access opens, okay? Real task scores over vendor benchmarks, every time.

And third, OpenClaw Enterprise launches with Red Hat, NVIDIA, and OpenAI, an open-source control plane for persistent agents paired with NVIDIA's new governance runtime, OpenShell.

Yeah, I mean, that's fascinating because that requires a lot of market checking and what are the models doing and a lot of agentic work, right? I suppose now with the agentic, you can just deploy a number of agents out there and really scale into ways that you couldn't before.

And a lot of this work, of course... we're intensely focused on for clients and so would it certainly encourage anybody who wants to kind of know more about the nuances of one model versus another to reach out to us because we'd love to talk about it but we also do continue to provide a free resource that's out there at governance arena which is a kind of a hybrid between OpenRouter and LM Arena for the broader governance community, right? And the idea behind it is you can plug in any governance question you have, or you can pick from one of our pre-canned activism situations.

For example, you can run, you know, Lululemon from this past summer.

and say, you know, hey, how would you land on this, right? And it's not uncommon in the activism situations or a shareholder proposal, whatever it may be, for the five leading frontier models to end up at three to two, right? And it might surprise you, like Grok we were finding, right? Grok had a real kind of activist tendency, right? Where it was like going after management in ways that might surprise people because you think, well, is that, you know, is that what I would have guessed?

This is an idea that was first originated LLM as a judge back in 2023.

The first paper was from 2023, Judging LLM as a Judge with MT Bench and Chatbot Arena.

It's a paper from a group of researchers at UC Berkeley, UC San Diego, Carnegie Mellon, Stanford, and MBZ UAI.

And the key idea of this is that they want to make a benchmark.

That number drops to around 60 to 65% when you're looking at cases that allow for ties and that allow for positional bias.

So that can actually make a really big difference in the numbers you report is the sources of those biases for the LLMs.

So where do these numbers come from? Well, this is one of my favorite little digressions here, which is that as part of sourcing this benchmark, they actually created an application called Chatbot Arena.

You may have found it at some point because it's still out there.

... and then it became a, a company.

Or LM Arena.

Yeah.

Um-

In doing heads-up experiences?

Yes.

And, and LM SIS actually did have a router project based on LM Arena ELOs, uh, which they never commercialized.

It's hard to do a company that does both because one company is taking data and selling it, and the other company really can't [laughs] by default.