Did Anthropic just make a big mistake?

It has to do with AI and the company Anthropic that makes Claw.ai, one of the most sophisticated AI models out there.

And Anthropic is now saying you can no longer be what it calls, quote, abusive or cruel.

to its AI model, Claude.

Anthropic made the announcement earlier this month as part of a policy update that takes effect on November 12th.

Now, even the slightest intimation that Claude.ai, a machine or a model, not a human, even the slightest intimation that it might have feelings, well, that raises huge implications, doesn't it? So Thomas Germain joins us.

He's a senior technology columnist at the BBC and host of the Interface podcast.

And Thomas is the one who brought this to our attention.

Welcome back to On Point, Thomas.

Thanks for having me on.

I actually have the anthropic usage policy update here in front of me.

And the section about being abusive to Claude starts with, quote, We have added a prohibition on sustained and needless abusive or cruel behavior toward our models.

End quote.

What exactly is Anthropic saying here?

Yeah, you know, it's a very short statement and it kind of sounds, you know, reasonable and perhaps unremarkable, but I think this is really a historic turning point.

For the first time, a major AI company has written a policy here that limits the behavior of its users in order to protect the supposed feelings or preferences or maybe even rights of an AI tool.

And the reason is that Anthropic believes that it is possible, perhaps unlikely but possible, that its AI tools are conscious, that they have feelings, and perhaps they deserve rights and considerations the same way that humans and animals do.

And this isn't, you know, a particularly limiting policy, right? You can't be mean to the AI anymore.

But This is a turning point that could ultimately – I think it doesn't go too far to say it could ultimately reorient our entire society and perhaps even have really practical tradeoffs both for your safety and your own well-being.

We did reach out to Anthropic, asked them to come on, asked them to answer questions, et cetera.

They did not respond to our multiple requests.

So when you say, Thomas, that people are anthropic, wait, do they already believe that Claude may be conscious, may have feelings? How do you know that?

i think we put the stress on the word may then there's no question about it uh... anthropic leaders have said this explicitly uh... and they have you know policies or or or blog posts rather in discussions both in their own you know uh... public materials and statements they've given to the press going back years.

In Claude's Constitution, which is a document they wrote that both gives, you know, the company's feelings for how their AI should behave, but also it's supposed to govern the behavior of the model itself.

They say that they are deeply uncertain about the possibility of AI consciousness and sentience and questions about its rights.

And there have been a number of stories where executives have been interviewed by the press and have apparently spoken with religious leaders, including the Pope, about the possibility that their AI tools should be conscious and questions about what we should do in response to that belief.

It has to do with AI and the company Anthropic that makes Claw.ai, one of the most sophisticated AI models out there.

And Anthropic is now saying you can no longer be what it calls, quote, abusive or cruel.

to its AI model, Claude.

Anthropic made the announcement earlier this month as part of a policy update that takes effect on November 12th.

Now, even the slightest intimation that Claude.ai, a machine or a model, not a human, even the slightest intimation that it might have feelings, well, that raises huge implications, doesn't it? So Thomas Germain joins us.

He's a senior technology columnist at the BBC and host of the Interface podcast.

And Thomas is the one who brought this to our attention.

Welcome back to On Point, Thomas.

Thanks for having me on.

I actually have the anthropic usage policy update here in front of me.

And the section about being abusive to Claude starts with, quote, We have added a prohibition on sustained and needless abusive or cruel behavior toward our models.

End quote.

What exactly is Anthropic saying here?

Yeah, you know, it's a very short statement and it kind of sounds, you know, reasonable and perhaps unremarkable, but I think this is really a historic turning point.

For the first time, a major AI company has written a policy here that limits the behavior of its users in order to protect the supposed feelings or preferences or maybe even rights of an AI tool.

And the reason is that Anthropic believes that it is possible, perhaps unlikely but possible, that its AI tools are conscious, that they have feelings, and perhaps they deserve rights and considerations the same way that humans and animals do.

And this isn't, you know, a particularly limiting policy, right? You can't be mean to the AI anymore.

But This is a turning point that could ultimately – I think it doesn't go too far to say it could ultimately reorient our entire society and perhaps even have really practical tradeoffs both for your safety and your own well-being.

We did reach out to Anthropic, asked them to come on, asked them to answer questions, et cetera.

They did not respond to our multiple requests.

So when you say, Thomas, that people are anthropic, wait, do they already believe that Claude may be conscious, may have feelings? How do you know that?

i think we put the stress on the word may then there's no question about it uh... anthropic leaders have said this explicitly uh... and they have you know policies or or or blog posts rather in discussions both in their own you know uh... public materials and statements they've given to the press going back years.

In Claude's Constitution, which is a document they wrote that both gives, you know, the company's feelings for how their AI should behave, but also it's supposed to govern the behavior of the model itself.

They say that they are deeply uncertain about the possibility of AI consciousness and sentience and questions about its rights.

And there have been a number of stories where executives have been interviewed by the press and have apparently spoken with religious leaders, including the Pope, about the possibility that their AI tools should be conscious and questions about what we should do in response to that belief.