69: Should we worry about AI in health care? – with Neil Lawrence

That's an interesting question.

I tend to see this Doom talk as being a reflection of two very serious problems.

One is over-concentration of power and power asymmetry, so reduction of power with individuals and increasing of institutional powers and questions over accountability.

And then the second piece is decisions being made by individuals automated processes or machines that we have no comeback on.

And I think when we hear that to whom talk, it's like those two things happening together.

But my feeling is, well, if those things have happened together, you know, we must have been in a position where they're happening separately on the way there and addressing those two pieces.

is always, you know, for years what I've felt to be the actual challenge.

And I'm afraid the doom talk is often coming from the people who are benefiting from those power asymmetries.

So they seem to use it as a distraction from the real problems, which is understanding how we deal with large corporate power.

And why do you think there's been an escalation in the last few weeks? Is it because of this famous Hugging Face case where the agents went rogue and all swarmed and tried to get under the skin of Hugging Face in a way that wasn't predicted? Is that the reason? Because there seems to be in the last few weeks, you know, very senior people either resigning from these companies and talking about it or talking about what they call in Silicon Valley the P-Doom level.

In other words, the probability of doom.

percentage that they have, which at the moment seems to be varying between 1 and 10, 10% or 50%.

The Silicon Valley bubble is quite cultish, and it's cultish around a certain form of intelligence, which is the certain form that tends to advance you in Silicon Valley.

So ability to deal with very abstract things like computer programming, capability in mathematics or chess or playing Go.

These are very rankable forms of intelligence.

There's a history also with eugenics.

I'm not saying that these people are eugenicists.

That there's a sort of simplification of intelligence that you see as characteristic of both.

And that simplification is that, oh, these abstract things are hard.

And what's very interesting is that the machine... quite understandably, is very capable of doing these abstract forms of intelligence, whether it was chess, cybersecurity, or more recently, there's been a very interesting story about solving maths problems.

There's two reactions I tend to find that people have for being surrounded by other people who are very capable, that you either end up with a sort of imposter syndrome because you're looking at everyone else and saying, oh, look how clever that person is.

Or some people actually believe that their form of intelligence is the real one.

And I'm afraid that at the top of these companies, some very successful people by their nature, their self-confidence is of that form.

So the fact that they tend to feel this is an existential threat is largely to do with the fact that the things we're getting machines to do are all the type of things they were very good at.

So imagine how that feels.