So, like, what is the use for you, as a media, uh, figure and podcaster? Like, what is the thing that you wish the models could do that they can't currently?

I, I think, um, because we're so-- First of all, every time I'm bear, say something bearish about the models, I, I, I put it into context that it, we're living in, like, an absurd timeline, and I am reacting to my, you know, close friends who are just like-- Well, you just had some of them on, and they're talking about, like, th- the singularity in two years.

Um, but I feel like we're so used to what these models are capable of currently that we ask these questions like, "Well, what is it that they can't do? Aren't they clearly already AGI?" It's like, no, we all have jobs.

That wouldn't happen in a world with AGI, right? Like, get, just get them to do something, um, uh, pretty-- Okay, so for example, I'm negotiating with a sponsor for next season or something, and, like, they ask for, you do, like, the back and forth there with the relevant context about how we think about our, uh, business and stuff.

It's, like, probably a one-hour horizon task for me or my general manager.

The models couldn't do it at all.

Um, or, like, let's say book a show in another city.

Are you still as convinced that, like, this is gonna be a major hiccup to getting us all the way to AGI? Or has, uh, have recent developments, you know, maybe any new models changed the way you think about that?

Um, so first of all, can you explain in broad brush strokes what is happening with AI? I know it's a massive question obviously, but do your best.

Um, well, I, I can explain it very concisely.

The models are getting better, and then we can n- be a little less concise.

I think it's...

Y- you're correct to point out that there's this huge discrepancy between what people are seeing in Silicon Valley and what people are observing outside.

Um, it's frankly because of how useful the models are becoming at certain kinds of things.

So by models, I mean, um, you've, you've seen ChatGPT.

Wait, robots can't pick up a cup?

What do you think that we've realized about human learning and human intelligence from architecting AI intelligence?

Hmm.

There's this really interesting thing we've seen where these AI models are making progress first in the domains that we think of as the archetype of, um, the- the- where humans have their primacy, right? So if you look at, um, Aristotle.

What does he say? What- what makes humans unique? Um, well, it's reasoning.

Humans can reason, other animals can't.

And what...

These models, these AI models, they're just not that useful, uh, if you try to use them for your work.

What, what else do most people not understand about the tension between China and the West in either direction?

That's like learn- like (laughs) learning from mistakes is, like, kind of high on the list in terms of how to become great at any specific task or initiative.

100%.

Um, and so then people will say, "Well, look, maybe the way, uh, we can l- they can learn from their mistakes," uh, uh, Jordy, is like, you can just tell it in the context, "Hey, you fucked up this way last time you were working for me.

Don't do it again." But I think the- this is at least an order of magnitude less efficient and less, um, less, uh, capable than the way humans learn.

So, the example I use here is imagine if you were trying to teach a kid to play the saxophone.

Um, but the way you had to teach this is, uh, w- you know, a kid comes into the room and they, like, try to play it cold, right? They've never seen a saxophone before.

They try to play a saxophone, and obviously it's not gonna sound the first- great the first time.

Yeah.

If we're all looking at the same data, why are there such vastly different perspectives on where this goes?

Hmm.

I think people have different philosophies around what intelligence is.

That's part of it.

I think some people think that these models are just basically baby AGIs already and they just need a couple additional little unhobbled eggs to go sprinkle on top.

Uh, things like test time, um, uh, thinking, so we already got that with 01 and 03 now, uh, where they're allowed to think.

They're not just doing... like, saying the first thing that comes to mind.

Am I getting that right?

(laughs)

... that sooner or later there will be a point in which a human is just unlikely to add anything sort of, like, counterfactually valuable, um, at least in a sort of, like, uh, uh, count the GDP numbers sense, right? Where horses right now, you can ride them for fun, but you're not...

uh, they're, they're not there because they are needed f- to move you around.

And it is obviously a big change.

Um, I, I have this intuition that humans have adapted to such diff- like, think, think about the world today and how many different transitions there have been, um, not just from the forge of our past, and even then, people underestimate how big a deal fire was or how big a deal tools were, the Agriculture R- Revolution, the Industrial Revolution, secularization of society, um, the buildup of states instead of having just, like, your local kin or whatever.

Um, if we've dealt with all of that, I feel like we'll do okay with material superabundance, um, but you're not that, uh, sort of, in the grand scheme of things, uh, economically productive.

And the other thing I think people gloss over, which, um, I think is important to the picture of what happens after but people are maybe uncomfortable about is transhumanism.

When in fact, what's really fun is that two people who respect each other, as I think we do, have, uh, a more peer-to-peer conversation than this famous person I'm trying to interview who's, uh, way above me professionally and intellectually and so on.

... than the people you interview?

So, um, if you have a Pdoom of 10% or 20%, that is, first of all, unacceptable.

The idea that everything you care about, everybody you care about could in some way be extinguished, disempowered, so forth, um, that is just an incredibly high number.

Just the, uh, but, like, let's say nuclear weapons is, like, a doom scenario.

If, like, if you're like, uh, "Should I go over to war with this country?" and there's a 20% chance that there's no humans around, um, you should not take that bet.

But, um, it's, it's h- harder to maybe express the kinds of improvements which are...

Uh, this will sound very utopian, but we do have peak experiences in our life.

I'm curious what for you the value of broad knowledge accumulation is in an age of powerful AI.