Finding a Precondition for Consciousness? The Emergence of Global Workspace in LLMs

Well, one thing that I did, you know, between then and now is I saw this paper called Verbalizable Representations Form a Global Workspace in Language Models.

And I thought, boy, there's a complicated title.

And it will make, in terms of the content, a really sort of interesting sort of theoretical analysis.

episode for us and so when you're away scott the cat does play and comes up with all kinds of ideas about these theoretical episodes so we need you here to bring us back down to like the ground otherwise we're going to be doing this theory stuff again because you know that's where my mind goes

I don't know.

It's interesting.

This is a good one, too.

It is a dense paper, so I am happy to contribute however I can to make it a little more accessible for folks.

But it is really, really exciting.

And there are a lot of implications for how we interpret and understand what's going inside large language models.

And I think that's kind of the key context that you need to keep in mind as we begin discussing the actual paper.

Okay, so let's just jump right into it.

We'll do my theory this week, and then next week you can bring us back down to earth with something.

All right, folks, so we're going to be talking about this really interesting development that I'm going to think of as another emergent characteristic in large language models.

And that's really the category that this is going to fall under.

You know, we've spoken in the past about about how large language models sometimes have these emergent capabilities, emergent characteristics, and this is one of them.

So again, the title of this paper, which is done by Anthropic, is called Verbalizable Representations Form a Global Workspace in Language Models.

And we're going to talk about what global workspace is in a minute because it's got really sort of specialized meaning.

But the paper is fascinating.

And the plan today is to sort of go through it at a pretty high level and then recommend that you folks read it and read some of the news articles about it.

But basically what Anthropic did was it went looking inside of one of its own LLMs, its high-capability LLMs, and it found something that had not been put there on purpose.

One of these, as we call them, emergent capabilities, which is a small, special set of internal, what we're going to call, thoughts that behave differently from everything else that the model does.

And, you know, really what most of the model does is it's going to run on autopilot, out of sight.

But this little set of what we're going to call thoughts can be read and described, and you can even figure out some reasoning from it.

And the researchers then sort of understanding that there was this inside of their LLM that they had not expected.