Cathy O'Neil

We do book recommendations.

Luce, do you have a book that you wanted to recommend?

Yes, there's one that I recently read, and it's called Weapons of Math Destruction by Cathy O'Neill.

I think the title in itself is already genius.

And it's basically how some algorithms that are used in big data can actually cause more inequality and eventually even threaten democracy.

Because when you go into the Matrix and you fill in your questionnaires and you play along, it, it, it kind of looks like as if you're being a bit churlish to say, "Actually, I have a really profound and deep rejection of this technology.

Everything about it hurts me." And I think that to open up a conversation about shame and shit, which is what we're talking about in therapy, but we're actually talking about, you know, the shame machines, as Cathy O'Neil calls them.

We're talk- Something really gets triggered on such a kind of delicate and deep level.

In the way that psychoanalysis is really interested in the delicate nuance and unspoken communications and unco- and, you know, the language of my unconscious being understood by you and not by me, and all of those complex human experiences just get obliterated in these questionnaires and behavioral techniques.

Yeah, well, um, it just reminds me of one of my favorite things to mention, um, which is that early on in the days when I was testing AI for the very first time, I asked it about myself, and it just gave me, like, lots and lots of inf-inf-- false information, and then I was like, "That's false.

That's not true about me." Or, "That's not true about Cathy O'Neil." I didn't actually say me.

And it was like, "Oh, I'm so sorry, blah, blah, blah, blah.

This is what we..." And it did-- And they spewed out other bullshit, and then I was like, s-like, "I'm gonna test something." And I was like, "Also, this true fact about me is not true." And it was like, "I'm so sorry.

Like, why don't we actually do something so we can practice for fighting Terminator? You know, and there's even a, a really amazing graphic that, uh, c- our colleague Lauren Smith made out of the big data work we did, where she looked at all the individuals and societal potential harms and, and graphed this.

And then she listed all the mitigations from a huge literature audit she did, like weapons of mass destruction from Cathy O'Neil, some Joy's work, our work, probably 25 things, and this was written in 2017.

And so back to school, I wish that more computer science faculty would incorporate problem set problems from those mitigations and, and mobilize our amazing young computer science students and those who are working with them to practice at mitigating challenges in AI, as well as working on problems in AI where there's a total absence of technologists working, like on social safety net or environmental justice in addition to health and other things.

So that's an area that I'm really excited about, sort of the missing, um, um, innovation cape teammates.