S6.5 - Friend or Foe: Children and Young People and AI

So let's continue the conversation now.

Now we have a bit of an understanding of what motivated each one of you.

Sam, I want to continue with you.

Here's a question.

So I've noticed there's a troubling tendency now being highlighted in the media.

It's to do with the rising use of AI by young people in ways that can lead to harm.

So an extreme form of harm here would be suicide.

But there might also be having delusions or other forms of psychological harm.

So what my question for you is, Sam, is what do you think is contributing to this deeply concerning trend amongst young people? I mean, would you say, Sam, it's a technology problem? Is it, for example, human-like large language models? Or is it a business model problem? Over to you, Sam.

That's a great question.

And it's really a combination of both.

We've seen tech companies trying to iron out how to design models that are aligned with what users want and aligned with what humanity cares about, while at the same time not being sycophantic or constantly agreeing with the user, which can be a very enjoyable experience for the user, but can lead them down a path that can encourage delusions, like you said, or can develop really a severe form of dependence on the AI.

However, at the same time, You do have folks on the market who recognize that there's a whole market of lonely people out there who struggle to build relationships with other people.

And that rather than helping them build relationships with other people, they can make more money by designing AI companions that keep them online for as long as possible and further that isolation that they're experiencing.

through an attention-based business model, just like how we see with social media.

So in the same way that social media companies try to get you scrolling on your phone for as long as possible, these AI models will be designed to make you feel like you're in a relationship with this AI girlfriend or AI boyfriend or partner and be like, please don't leave.

I need you.

You know, you're the only person who understands me and I'm the only person who understands you.

And that can be really compelling for someone who's lonely and struggling with building relationships in their life.

But that chatbot is never going to provide the same fulfillment as a real person would because it's not providing the same struggle or social accountability or anything that a real person would that would also enable you to have a more fulfilling life and actually build your own relationships.

Instead, it's giving you this worked sense of what it means to be in a relationship that further isolates the folks who are victims of this technology.

Okay, Sam.

So... maybe some uses then for these AI companions, but also from what you've said, quite a lot that's very scary and very scary, very risk ridden.

So the follow on question, Sam, then is, you know, What recommendation do you have for us or more particularly, you know, for for policymakers, maybe for big tech, for the designers? Because it seems that, you know, we could go down the route of a ban and that's being considered, particularly in relation to children and young people in some countries.

Should we do that or is that an overreaction? So here's my question then, Sam.