OpenAI's Hugging Face debacle makes great case for open models

You know, it seems like kind of a bit of a careless thing.

But, you know, as we were kind of alluding to another story you wrote this week, that, you know, this whole thing's wild, and it's an indication that maybe some of the things that, you know, anthropologists are learning about mythos' capabilities might be true, but...

Right.

I think a story you wrote talked about how one cybersecurity expert basically said, yeah, but, you know, you've got to have all these preconditions.

Right.

Like we were talking about in order to make this work the way it did.

And so the likelihood of it happening isn't necessarily as great as, you know, the sky is falling.

Is that is that kind of a correct optimization?

Exactly.

There's three really key points that seem to be missing.

And the first one I already mentioned, it's that the guardrails weren't enabled.

So they didn't have these safety guardrails in place.

So if you tell the agents to go find an attack method and you take away all their guardrails, that's what they're going to do.

And at the same time, it's interesting because then we also know that likely OpenAI's models with guardrails enabled refuse to help hugging face.

So they're doing what they've been trained to do.

They're saying, no, we're not going to do that versus the no guardrails.

Sure, we'll find any attack method we can.

And another thing that...

So the cybersecurity expert, his name's Renato Marino, and he's the chief research officer at Morphus Labs.

And he pointed out something that we've pointed out.

I know Tom has written a lot about this, so have I, that AI companies touting their models, autonomous bug hunting and exploit finding abilities also is kind of a marketing win for them.

It shows how powerful they are.

So...

OpenAI doesn't really lose anything by saying, oh, yeah, it was our models that powered these agents.

Right.