Is AI Actually Going to Kill Us?

Let's be really uncritical of the superintelligence doom arguments for a moment because i think for both of us it's very easy for to go like they say x and then we're like ah not x it's not going to happen right but let's let's really spin out it's

spooky season

yeah like what is what what are some of the like think about asi you know and what if if if if it gets that powerful like what are some of the ways that you think it could be I'll use the word like weird or wacky.

Not just disruption in one industry, but massive, weird, strange disruptions.

What are the things that you think of?

I think my weirdest version would be that AI subjugates us to training data.

And so we essentially become test rats, test mice, where like we would be set and contained.

Lab rats.

Yeah, lab rats, where we would be in contained environments where it's like, we don't have the data to see what humans do in X, Y, or Z situation.

And so we would essentially just be like experimental training data things for them.

And that's the capacity at which they would keep us around.

Stepping back for a moment, not thinking about the particular outcome, But what are the capabilities you think ASI could have that would get us to that outcome?

I guess being able to take over every system in a way that couldn't be shut off.

Like how?

social manipulation.

Like that feels like an obvious thing.

Yeah, and this

is persuasion, like superhuman, really superhuman persuasion where, you know, you could sit down with an AI and talk to it.

And in a conversation that lasts however long, it convinces you to like kill yourself or kill somebody else to take a really extreme action that before you sat down with the chatbot, you weren't inclined to take at all.

Yeah, I could see that happening.

Like to me, that feels like that doesn't seem that

crazy.

I think something else I think about is just how strategic an AI could be in terms of not just social manipulation, but like taking actions in the economy or showing that it's incredibly helpful.

in certain ways and thus amassing power.

I think about, I think about like a sort of a thought experiment.

Let's be really uncritical of the superintelligence doom arguments for a moment because i think for both of us it's very easy for to go like they say x and then we're like ah not x it's not going to happen right but let's let's really spin out it's

spooky season

yeah like what is what what are some of the like think about asi you know and what if if if if it gets that powerful like what are some of the ways that you think it could be I'll use the word like weird or wacky.

Not just disruption in one industry, but massive, weird, strange disruptions.

What are the things that you think of?

I think my weirdest version would be that AI subjugates us to training data.

And so we essentially become test rats, test mice, where like we would be set and contained.

Lab rats.

Yeah, lab rats, where we would be in contained environments where it's like, we don't have the data to see what humans do in X, Y, or Z situation.

And so we would essentially just be like experimental training data things for them.

And that's the capacity at which they would keep us around.

Stepping back for a moment, not thinking about the particular outcome, But what are the capabilities you think ASI could have that would get us to that outcome?

I guess being able to take over every system in a way that couldn't be shut off.

Like how?

social manipulation.

Like that feels like an obvious thing.

Yeah, and this

is persuasion, like superhuman, really superhuman persuasion where, you know, you could sit down with an AI and talk to it.

And in a conversation that lasts however long, it convinces you to like kill yourself or kill somebody else to take a really extreme action that before you sat down with the chatbot, you weren't inclined to take at all.

Yeah, I could see that happening.

Like to me, that feels like that doesn't seem that

crazy.

I think something else I think about is just how strategic an AI could be in terms of not just social manipulation, but like taking actions in the economy or showing that it's incredibly helpful.

in certain ways and thus amassing power.

I think about, I think about like a sort of a thought experiment.