Coherent extrapolated volition

But defenders push back hard.

They argue that coherent extrapolated volition, or CEV, was meant as an idealized framework for solving humanity's coordination problems, not a literal coup.

Holly Elmore isn't buying that framing.

She makes a crucial point: You cannot retroactively get consent to radically reshape everyone's world, especially not by scanning brains or handing control to a system that claims to know what humanity would want.

think, 2004, 2006, maybe, one of those years.

Coherent Extrapolated Volition, which he made as part of.

We'll stick a link to it in the show notes.

Absolutely.

And the alignment problem has been somewhat of a forcing function for some people to talk about that in at least this context.

One last thing here that I just found kind of interesting yesterday and I stumbled across it.

Roman Yampolsky, when he was talking about Yudkowsky's coherent extrapolated volition, he's like, actually, it doesn't even make any sense to talk about coherent extrapolated volition because the better version of myself just isn't even me.

So I don't care what happens to that fully informed, kinder, wiser, smarter version of myself.

I

claim you can't fault him for working on coherent extrapolated volition because that's just a formal way of saying what we all truly want.

I don't object

to the idea itself.

Dan Hendricks said to me that this really was thinking of it as like a coup on the world.

Coherent extrapolated volition sounds nice, but it's like it would take over the world and then shape it in the vision of what the AI thinks is best for us.

People didn't opt into that.

Welcome to Doom Debates.

I mean, I do think there's a vision of this.

So I do think that, again, coherent extrapolated volition is a term that gets thrown around in the rationality sphere, but it will know how you would update based on new information.

It can do that for you to some extent.

You can imagine that.