Stochastic parrot

So I figured, let's start with that, right? Let's start with just setting out the case, you know, in a way that pulls no punches, right? But, uh, actually, uh, uh, uh, next week we're going to be reading, uh, papers that take a much more skeptical perspective.

We're gonna read the Stochastic Parrots paper.

Hmm.

We're gonna read AI as a Normal Technology.

She was co-leading Google's ethical AI team.

She was the most senior Black woman in that department, and in 2020, she co-authored a paper with Dr. Emily M. Bender titled, "On the Dangers of Stochastic Parrots."

A very famous paper now.

Yeah.

And that paper has since become one of the most cited papers in all of AI research.

What was so incredibly dangerous about what they were saying that a trillion-dollar company felt they had to silence her?

The stochastic parrots paper was dangerous because it punctured the illusion of the machine mind

It broke the spell?

Because when you start talking that way, then you're, um, a- ascribing accountability not to the people building the artifact or the, the, the city that gave them the permit to do so or the improper kind of like materials that they used or something like that, but somehow, you know, you forget that there are all these people and entities involved in bui- in building this thing, right? So the only person I saw, um, actually, um, asking the right questions was Emily M.

Bender, who's the co-first author of the Stochastic Parrots paper that you referenced, and she said, "First of all, I don't believe all of this, uh, uh, you know, hype about, uh, GPT-2.

And then when OpenAI built GPT-3, it was even more of the same.

Um, and she said, "I don't believe it.

Um, and also this, all this data center talk, et cetera, is a distraction.

In fact, actually, there have been articles seeded by these companies that are saying that, that the, the data centr- uh, center opposition is a distraction.

Um, the, the biggest, um, coordinated talking point right now actually is that my paper, Stochastic Parrots, is a distraction and set us back years, is what they say.

Years in, um, AI regulation 'cause we didn't think about extinction.

I had been working with a lot of folks who not only were very concerned about the sorts of things that you just mentioned, but also were very skeptical about how capable AI was gonna get.

So some people may be familiar with a research paper called Stochastic Parrots, who, whose authors have continued to work in this and, you know, their views, I think, differ from each other and have evolved over time.

But basically, the idea was something like, these systems are merely good at seeming clever and are not going to be even capable enough to do a ton of economic valuable work.

And I actually helped to create the research conference where that paper was published and was the program chair when we published it, and I, I, I don't think I was fully persuaded of that view, but I certainly was among people who were skeptical.

Uh, next one is, I wonder what the next guy says.

[laughs] Lattice Washroom, Seth, Jenny Kim, The Stochastic Parrot, Ira Chan, Jonathan King, all one name.

To show you love AI, put an upside down pineapple on your cubicle or office door.

Okay, I do know this joke, and don't actually do that.

Okay, carrying on.

Let's see.

Uh, Jenny Kim, The Stochastic Parrot, Ira Chan, Jonathan Kingson, I Beautiful Functional User Documentation.

Adrian at Kappa.ai, empty name to protest the new name length limit.

Companies whose products, after all, get to be hailed as all-powerful, who get invited to the White House to decide how they should be regulated, that is not at all, and who get to prepare for forthcoming IPOs as the harbingers of superintelligence.

Now, one term in particular drives the AI industry absolutely crazy: stochastic parrots.

We'll get a bit more into the definitions and background in a bit, but stochastic parrots is a term coined by four leading AI researchers in a paper in 2021, and it essentially holds that large language models are not intelligent, but are rather systems that output material based on what's in their training data.

Wow, shocking, I know.

But while LLMs and technologies described as AI today have become more sophisticated and complex over the last five years, the term remains a fundamentally accurate way of describing what LLMs do, yet the industry hates it like the plague.

AI industry folks absolutely lose their minds over the term, and they often bring it up unprompted just to try to debunk it again and again.

When the AP reminded people that its style guide insists on not anthropomorphizing software programs, for instance, a s- direct descendant of the stochastic parrot's lineage and influence, industry boosters were furious.

So all this left me wondering, why? Why does this particular term have a unique capacity to enrage AI industry folks? I had a theory, a theory that even more so than other critiques about, say, shaky financing or AI's dubious functionality in a lot of cases, stochastic parrots undercuts the very premise, the foundation that the AI boom is built on, that tech companies are building a machine god of infinite power and potential.

Timnit, you might recognize her name.

She was fired from Google back in 2020 for the Stochastic Parrots paper, basically, which has now become this huge... fight in its own right of people getting mad about the idea that the AI is not actually a person, which is tough for some to swallow apparently.

But yeah, TESCREAL stands for transhumanism, extropianism, singularitarianism, cosmism.

rationalism, effective altruism, and long-termism.

Right? So you would automatically slow them down and have to make them accountable for something.

Back in 2021, you co-authored a paper titled "On the Dangers of Stochastic Parrots." Google had approved it initially, then it was being reviewed.

There were parts of it that were in dispute.

You said, "Look, if you want me to remove my name from this, I'm not gonna be a part of this." You ended up decamping from Google.

Uh, that paper was later presented at the 2021 ACM Conference on Fairness, Accountability, and Transparency, and people still reference it.

Can you briefly explain for someone who's never heard of stochastic parrots before what it has to do with the AI we're using today, what it means?

Um, stochastic parrots is, is, is, um, a metaphor to help people understand what large language models do, and large language models are trained on vast amounts of textual data on the internet, trained to output the most likely sequences of text given their training data.

They power most of the chatbots that we see today, whether it's Claude, whether it is ChatGPT.

And I immediately, you know, found some amazing papers, you know, by these amazing people.

researchers, like specifically, I think a good place to start for anyone who doesn't really know anything about this story would be the Stochastic Parrots paper written by Emily Bender and Timnit Gabriel and Michelle Mitchell.

And it's just kind of a great framework for all of the problems that we now see with large language models.

And there's, you know, a great, you know, so it's a great paper.