It's rather unnerving.

Well, to be honest, this is nothing new.

This kind of thing, these exact same statements, the 10% risk of human extinction and so on, have been being predicted for decades now.

And we haven't certainly... haven't seen it happen.

We haven't seen that AI systems are getting closer and closer to killing off all of humanity, which would be quite a feat, to be honest.

But there are certain risks that are coming to the fore now that I think do need to be addressed.

And what are those risks? Well, there's many risks that have been around for a long time, such as the risk of AI systems sort of polluting the internet with misinformation and disinformation and many more.

And Melanie, it's not keeping you up at night, but perhaps the idea of the lack of rules might be a little bit concerning.

I like to go to my grandma Laura's house, well, because she gives us candy

... which was a neural network that could take, um, text and turn it into speech.

It wasn't very good at all compared to what we have today, but, um, it was...

it learned that from, just from data.

There were no rules programmed into it, and so that was seen as, like, a really interesting kind of advance in the neural network world.

They're just too complex.

Trillion weights.

And so the scale is just unimaginable, the s- how large they are and how these things are all interacting with each other.

But just as a teaser, could you enunciate what are those six and say a little about them?

Sure.

So the first one is to be aware of your own anthropomorphic cognitive biases.

So we tend to project human likeness onto things that talk to us in fluent English, so people very much think that these models have human-like qualities when maybe they actually don't.

The second one's a very common sense one for scientists.

Be skeptical of hypotheses and develop control experiments.

That's just like Science 101, although I'm not sure how often it's really followed through in science.

For people who haven't heard our earlier conversation, what was your draw to this field? And if you were starting out today, do you think you'd have the same kind of curiosity?

Okay, so Professor Mitchell, tell me more about how AI, uh, is helping us understand better human thought through, I guess, modeling what we underst- know of human thought through these systems.

Yeah, I think that, that AI throughout its entire history has been teaching us about human thought, both by showing similarities and differences, and also by sort of questioning our assumptions.

So for example, back in the 1970s, many people thought that, uh, a computer that could play chess at a grandmaster level would obviously need to have a human level general intelligence.

That chess would be a good benchmark for that.

But of course, then we had these models like IBM's, uh, Deep Blue, which beat Garry Kasparov at chess, became better than any human without anything like general thinking.

Now we're sort of facing that same question about fluent conversation.

Hmm.

That's sort of been known as the Turing test.

Why do you think the mechanics of statistical pattern matching across billions of parameters still look so different from the way human minds form conceptual analogy?

Oh, wow, that's a hard question.

You know, there's a big question that whether these systems are interpolating between things that they've learned and things that they're asked, or whether they can actually extrapolate, you know, do something that they have never seen anything close to in their training data.

And I think that Making interesting analogies is one of the things that we humans can do that, you know, we haven't really seen in our quote unquote training data.

And I don't think machines are there yet.

They are still interpolating.

And when you have a huge sort of corpus of stuff to interpolate from, namely all of human digital writing, digitized writing, You do really well.

But as a practitioner, we're really interested in what you think about it.

This is something that you think is maybe lacking in the AI community at the moment.

So here's one example from my own work.

There's a very widely used benchmark for abstract reasoning called the Abstraction and Reasoning Corpus, or ARC for short, which consists of, uh, a bunch of little puzzles that involve re-reasoning about spatial concepts like inside versus outside or, um, above versus below or things s- same versus different, things like that.

And this benchmark has become quite Popular, a-and many tech companies have adopted it so that they'll post their results on this benchmark in their announcement for every new model they do.

Say like, "Oh, we, we surpassed humans, human performance on this benchmark." So my group actually went in and said, "Okay, you've surpassed human performance, but are you actually-- is the model actually doing the kind of abstract reasoning that the benchmark's trying to test?" And we were able to do some probes of the reasoning of these models, and we found out that in many cases it's solving the puzzles correctly, but using unintended features of the puzzles to get the right answer.

If I'm trying to get it to reason using, like, there's a square that's on top of a cube or something like that, it using those features, well, maybe it's actually using some counting of different colors that wasn't intended to be part of the solution, but actually by accident, uh, works.

And we found that in many cases that's actually happening because these models, they do so much computation in their reasoning, um, process, they're able to find these very unhuman-like features to solve the, the problem.

So if we mismeasure intelligence with these benchmark tests that we use for AI, what are the consequences for, for science and for how we build these AI systems?

Okay, so Professor Mitchell, tell me more about how AI, uh, is helping us understand better human thought through, I guess, modeling what we underst- know of human thought through these systems.

Yeah, I think that, that AI, throughout its entire history, has been teaching us about human thought, both by showing similarities and differences, and also by sort of questioning our assumptions.

So, for example, back in the 1970s, many people thought that, um, a computer that could play chess at a grandmaster level would obviously need to have a human-level general intelligence, that chess would be a good benchmark for that.

But, of course, then we had these models like IBM's, uh, Deep Blue, which beat Garry Kasparov at chess, became better than any human without anything like general thinking.

Now, we're sort of facing that same...

the question about fluent conversation.

Mm.

That's sort of been known as the Turing test, you know, can a machine fool you into thinking it's a human through conversation? And it turns out, and this, I think, was a big part of Kyle and, uh, Anna's, um, paper that you've been talking about, was that language, generating fluent language is not the same as thinking, that it can be done without this kind of more, uh, all these different kinds of mathematical, physical, et cetera, cognition.