Google Gemini and AI bias

yeah so i think there's a we've talked about intellectual property and we've talked about um um rights and hearing stuff before um the thing i thought was interesting to start with is that google's gemini model previously known as Duo, previously known as Google Workplace Chat, previously known as Chat, Google Chat for Workplace.

Google's Gemini Pro Plus Enterprise model, which is multimodal and does images.

And so it turned out that if you typed in, give me pictures of Nazi stormtroopers, you got a white person in Nazi uniform and a black person and a Chinese person and an Asian woman and so on.

Oops.

Which brings up so many good questions.

Yes.

And so I wrote something in, I think, 2019 about AI bias.

And the sort of tempting thing, particularly if you're the kind of person that thinks you've made a devastating retort if you say tech bro, is to say, well, this is because they don't have an appropriately diverse bunch of people working on these models, and it's all white people who went to Stanford in their 30s who don't know anything about the world.

And there's about 1% of truth in that, but it's mostly just a really stupid thing to say.

To understand why, let's go back to the AI bias problem that people were thinking about four and five years ago, which is you are using a system that is extracting, interpolating, inferring patterns from data and what's in the data.

and there may be things in the data there may be patterns in the data that are true but that you don't want it to use and there may be patterns in the data that are true in the data but do not actually reflect the nature of the world so that it may draw it may draw a conclusion that is in fact factually correct but you don't want it to use and you may not be able to tell it's doing that And you may also, it may also draw a conclusion that is in fact factually incorrect, but that just happens to be that's what's in the data, that's something, there's something in the data.

And the kind of a generalized way you can think about this is that you kind of worry about what happens if these systems work and you worry about what happens if they don't work.

Because sort of the next step is, you know, what happens if bad people do stuff with this and what happens if good people use this and don't understand what's going on.

And so it's very easy to think of a bunch of ways where, you know, parole decisions or hiring decisions say there might be stuff in the data that you don't want it to use.

There was a famous anecdote of Google trying to build a machine learning system to look at resumes because they were hiring at such scale.

And the majority, vast majority of the people that they hired, so the majority of successful hires were men.

Mm hmm.

So then it would look at a resume, and if it wasn't a man, it would downrank it.

And it wasn't doing that because it was flagged man or woman.

It was doing that for things that women tended to do and men tended not to do, like subjects that they might have studied or societies they might have been members of or sports that they might have played.

The language they use, the words they use.

Yes.

And indeed, the language and the way they talked about their achievements would tend to correlate with being a woman, which tended not to correlate.

They didn't tend to have the kinds of words that men tended to have, which were the simple set of successful resumes.

So, this creates a whole bunch of problems if you're in a regulated industry and you're not allowed to use gender as a factor.