So I guess if we look at what an LLM is, actually, under the hood, and what is it about what an LLM is that translates into the things it's good at and the things it's bad at?

So, good question.

I think, I mean, I've tried it a couple of times to kind of answer it, and I have various forms of answers to this question.

However, I feel like I don't think this question can be answered is kind of where I've come to at the end.

Because like, theoretically, a model that is able to recursively loop can, however slow, be quite good at solving things like cellular automata or can it be a Turing kind of machine, right? So as a consequence, is it possible for it to theoretically do anything? Yes.

The problem is that practically, even though, like, sort of a friend a couple of years ago on Twitter, I think, proved that Opus could act as a Turing machine.

You can fact check me on that, but I think it was two years ago.

But there just seem to be some incredibly powerful incredible like impracticalities to that.

Like how... what are, what are... what happened there in that and, and how would one begin to deal with those sorts of issues? Uh, h- either as a, as an individual managing a group of agents, or I guess as an institution, uh, [laughs] that is attempting to design a system such that you don't get fraud, [laughs] like as an emergent property.

Right.

I mean, it's a, it's a fantastic...

Like, so this is one of the things that I am obsessed with at the moment, and like I have three or four new sets of experiments that I've done and will come out in the next sort of few weeks, et cetera.

Um, uh, let me start with sort of the, the s- the why of this, right? We, we said a little bit about how models are basically y- whatever, you know, a Memento style graduate student who was always on his day one, like really smart RA, et cetera.

And the good thing about this graduate student is that like, you know, you put him in a economics lab, you put him in a mathematics lab, you put him in a paleontology lab, they'll, they'll do really well.

Like it...

... what's the st- [laughs] what does the state of the art look like in terms of, uh, dealing with this kind of thing?

Are we talking about a real difference there? Is, is this a, a really existing or potentially existing thing, or is this the sort of the conjurings of our most dystopian imagination?

So there are two aspects to it.

So first of all, it's real, right? I mean, there is, there is...

there should be absolutely no confusion about the fact that what we have today can do things that seem miraculous even to people who are building it or who are using it daily on a regular basis.

I think that is just fact.

However, a curious problem that we have is that we lack the vocabulary to describe the things that it is doing without invoking things like will or consciousness or sapiens or intentionality or agentic behavior, all of which are terms that apply to humans or we apply to each other, we apply to animals, and they mean something to us because they come embedded with, like, billions of years of, you know, evolution that we've all gone through and sort of we can kind of understand.

When I say, like, that horse behaves as if it's going to do something, we are ascribing an intentionality to that animal that we can then do something about.

Can you tell us what that means? [laughs] What is the, the significance of what's just been released out of China?

How do we do both things at once? That's the real question to me, not that you don't use AI, but what is it that you have to invest in to ensure that people have what I'm calling constitutive competency over time?

Eric, you want to jump in here? Maybe one thing to think about here is that like, you know, there is a set of tasks or set of questions that were asked to the AI as well as a bunch of doctors.

We saw the outcomes, you know, these were radiology, maybe diagnostic accuracy, a few of these different things.

And you saw the outcomes and you're saying like, this was right, this was wrong.

Two things strike me in this.

Number one is that like, I don't know whether the physicians who worked with AI are actually any good at working with AI.

In the sense that any consequential decision I have made in the last three, four years has been with the aid of AI.

I agree with you, it's not the norm, Azeem, but there are some wonderful examples that we can draw from and build from to start to recognize that in order to change our educational systems, to start to orient toward the felt capacities that are necessary to develop those kinds of intuitions, there are some that are working and that can be scaled.

How about you? What's a great

example for you? There's this essay that I wrote, I think, three years ago.

It basically said, like, computer used to be a job that people did.

Like, when we went to the moon first, there used to be, like, a room full of people who were computers.

They would compute things.

And now it's machine.

That, like, you just, you do everything on the machine.

And then you can decide how many resources you actually want to give that that agent do you want to give it your email logins do you want to give it your logins to your paypal or or not and you know perhaps don't do that at that point so that's kind of quite a quick way to get started you sync it up with a slack channel or a whatsapp channel there are there are instructions although it's pretty easy if you download the mac installer as well to do it that way around let's talk about securing it what do you do about securing your