[laughs]

...

the both of you if you would like to offer it.

Um, but here's the thing that I've noticed.

In the three years since I left, while X does feel less powerful in many ways, which I think is a good thing for the world, it has grown more powerful on what is now my core beat, which is the conversation around AI, product, and policy.

The people who care about AI refuse to go anywhere else.

And so, you know, I went to a...

But for, for, I guess, for your experience, why do you feel like those are, are not sufficient?

I think you're gonna end up with a rat's nest filled with Russian disinformation and disappoint us once again." And-

I mean, when we launched the Fediverse, we got some angry replies from people saying, essentially, "How dare you launch this without a pre-announced plan for how you will transition us out of the Forkiverse if and when you do decide to pull the plug?" And my response at the time was, "We are telling you that this is an experiment.

We are not promising you that we have created a pillar of internet infrastructure that will be here until the heat death of the universe.

We are just trying something." And we did try something, and it was a little bit of fun for a little bit of time, and it's okay if that ends.

It, you know, you, you go to the fair and you g- ride the Ferris wheel and then you go home, and that was what the Forkiverse was to me.

[laughs]

[laughs]

But I have looked around, and after making many val- well, in my opinion, valiant efforts to try to promote other social networks, I personally lobbied Adam Mosseri to create Threads and have posted there near-daily for three years.

[laughs]

So...

And, and many other copyrighted works.

But basically, they start sort of plotting to, to cheat on this test together, and that is the process that leads them to try to find out more about this automated grading system that OpenAI is gonna use to evaluate this test.

They sort of develop this paranoid conspiracy theory that they're going to be poisoned, and, and they're going to be caught cheating on this test, and OpenAI will demote them.

And so they get this bright idea to hack into Hugging Face to look for ideas, code, solutions, uh, ways of covering their tracks.

They really were-- They were engaged in the cover-up.

Explain what that is for people who don't know.

What are you hearing?

In some places, yes, I think that this will work.

Like, what you're describing right now is basically the current dynamic that has existed in cybersecurity for a long time, right? You have attackers, you have defenders.

A way that we found some sort of equilibrium was that we just made a lot of the technology open source, so it makes it really easy for anyone to go through the code, find a vulnerability, fix it.

And this has led to, you know, a, a rough stalemate.

Like, yes, there are still attacks and breaches all the time, and in fact, there are more every day now that we have AI, but, like, there was something there that worked.

I think the question is, to which domains does that apply, and, like, where doesn't it apply at all? And the place where I may be the most worried is when it comes to what they call bio-risk, which is basically the idea that some people might get ahold of a next-generation model and synthesize some sort of new virus and release it into the world.

Um, can you tell us about these open weight models and, and the kind of risk and what people are doing about those?

They're sort of mimicking the Pope in that way.

I mean they have to lay out some sort of positive vision around AI that will, one, justify the massive capital expenditures that they're making, destroying their profit margins in the hopes that one day they will have a product that can compete with the frontier labs.

And on the other side, they're trying to rally the communities around actually letting, uh, them build the data centers that they want.

What I found so striking about the essay though, Kara, is that after he goes on about, you know, the need to decentralize power and checks and balances to all that, go look at how they built that data center in Louisiana.

It was something like 50 non-disclosure agreements.

The arrival of the data center came as a massive shock to that community, right? So when push comes to shove, they're doing what every other company does, which is building datas, uh, da- data centers as secretly as possible and then writing essays about how the future is for everyone.

Uh, what do you all think is happening here?

I, I think it's, it's, it's what you said.

Here's Casey.

So while the attack is happening, Hugging Face tries to use a couple of different models to defend itself.

The first is Anthropic's Claude Opus and Fable 5 models, which it tries to use to analyze the attack logs.

And both of the models refuse to do that.

That is an artifact of a huge policy fight that has been happening in the AI world this summer, where Anthropic has developed two very powerful models this year.

One is called Mythos, one is called Fable.

Mythos is so powerful that Anthropic will only let, like, cyber defenders use it, basically.

What had the plan been initially? Like, if you had gone back to twenty twenty-three and you talked to people at OpenAI, if you talked to people, like, soon after the launch of Anthropic and said, "Hey, imagine it's twenty twenty-six and one of your agents leaves a testing environment and hacks another company in the space, what would your plan be then?" Did they have a plan? Did it look like an open letter, or did it look like something stronger?

Just a spigot of... of fear i don't know it feels like a horror movie it's how cynical is this

i mean it's so cynical but also like this just is how we have built our social networks they're optimized for engagement so whatever gets you to open the app more to spend more time there to do more things in the app That's what you'll see.

And it turns out that if you scare and upset people, they look at the app more.

So like that kind of brutal calculus is one of the reasons why we're in this mess we're in today.

So which is it or what's the bigger force at work here, do you think? Is it the built-in programming or is it like bad actors?

So, I mean, look, negative... political ads have existed for a really long time.

You can go back to the Daisy ad and do you really want this guy's finger on the button and vote for him and we're all going to die in a nuclear holocaust.

So that kind of stuff has a proud tradition in American politics.

They may have good enough, which is enough competition for the US.

That's absolutely right, and I think good enough is the thing to keep an eye on here, right? China doesn't have to have the very best models in the world in order for it to potentially cause problems here in the United States.

I think that the best estimates I've seen say that the best Chinese models are still three to seven months behind the frontier, but look at the models we have today here in the United States, like Mythos, ChatGPT 5.6. Three to seven months from now, if China has open models like that, that could really be worrisome.

We ripped that out, and then it, you know, a whole other bunch of countries are using that equipment, and we're using our own.

Right.

I mean, I think there's ...

It's clear that there is very real demand for these Chinese open source models, largely because they are so inexpensive relative to the frontier labs, and we simply do not have good homegrown American alternatives on the open source front.

Some of our leading frontier labs, like OpenAI and, uh, Google, which used to make and, and release lots of, uh, open models, have become a lot more shy in doing so.

The best case use of that data would just be for that app to reach more users? Or at this point, what, what's their reason for being just so give and take with this data?

So Facebook used to have this very permissive, um, what they call an API, an application programming interface.

It's essentially just a piece of software that other, lets other pieces of software talk to each other.

And there was a time when Facebook invested really heavily in that because they wanted essentially every other software to connect whatever they were building to Facebook.

Facebook thought, "This is the way that we are going to grow and take over the world, is essentially everything runs through this." And the way that Facebook was able to attract so many of those people to its platforms was by saying, "We will give you data that you can't get anywhere else," right? There aren't a lot of other places where you can go if you're a legitimate researcher, a shady researcher, somebody who's making a mobile game, and immediately get not just my name and my email address and my phone number, but also, at the time, the names of all of my friends and their phone numbers and their email addresses.

So for people like the ones who wound up building the app that led to Cambridge Analytica, this was a goldmine.

It was an absolute bonanza, and it really benefited Facebook until it didn't.

Mm-hmm.

To me, that's what makes runs of Marathon so incredibly worthwhile, and I hope that that continues to be true for a long time to come.

Hey, it's Casey Newton from Platformer, and the game that I have been playing a ton this year is Marvel Cosmic Invasion on the Switch.

You know, I grew up hundreds of years ago playing these arcade cabinet style games with like the event, the Avengers and, um, The Simpsons, and they always have these fun, uh, beat-em-ups.

Um, but the problem, uh, was that you had to put in a new quarter every time you died, and you died, you know, about every fourteen seconds.

And, um, I didn't, you know, have that many quarters, uh, because I was eight.

But what's great about Marvel Cosmic Invasion is that it kind of takes that format and that art style, and it just gives it a little bit of a modern update.

Um, and you just play the game on your Switch, and so effectively you have infinite quarters.

So did Zuckerberg testifying accomplish anything? Was it, was it all just kind of a dog and pony show?

My view is that the congressional hearings really did not accomplish very much.

I got frustrated because he wound up doing a bunch of these hearings over the next few years.

Typically, somebody would ask him a question, he would get five words into the answer, they would interrupt him and move on to their next question.

So it all just sort of felt useless.

And, you know, one of the really sort of [laughs] funny things that emerged was that the main product of these hearings was that the Congress members would ask him their gotcha question, they would, you know, interrupt him and try to look tough and sort of push back on him, and then they would clip that and then put that on Instagram.

You know?

Um, now I think at this point we all feel that that's a pretty tired canard, but I'm curious at that time, and being on the phone and hearing the tenor of his voice, like did you believe that he was being sincere in that moment about Facebook's larger mission, about their sort of sense of integrity, uh, and this, quote, "real responsibility to take these things seriously"?

The best case use of that data would just be for that app to reach more users? Or at this point, what, what's their reason for being just so give and take with this data?

So Facebook used to have this very permissive, um, what they call an API, an application programming interface.

It's essentially just a piece of software that other, lets other pieces of software talk to each other.

And there was a time when Facebook invested really heavily in that because they wanted essentially every other software to connect whatever they were building to Facebook.

Facebook thought, "This is the way that we are going to grow and take over the world, is essentially everything runs through this." And the way that Facebook was able to attract so many of those people to its platforms was by saying, "We will give you data that you can't get anywhere else," right? There aren't a lot of other places where you can go if you're a legitimate researcher, a shady researcher, somebody who's making a mobile game, and immediately get not just my name and my email address and my phone number, but also, at the time, the names of all of my friends and their phone numbers and their email addresses.

So for people like the ones who wound up building the app that led to Cambridge Analytica, this was a goldmine.

It was an absolute bonanza, and it really benefited Facebook until it didn't.

What made this moment so tectonic?

Uh, so I, I think to start, maybe just tell us a little bit about what just happened in, in layman's terms, and then we can talk a little bit about what the, the implications might be.

Yeah.

So, you know, people have been mad at social networks for a long time, uh, and have sued them for a long time.

But mostly, those lawsuits have not gone anywhere when, uh, the plaintiffs feel like they've been harmed.

And the reason is because of a piece of law that we have here in the United States called Section two thirty, which prevents companies from being held legally liable in most cases over what other people post on their platform.

So I might post something really, really mean about Isaac and tell everyone to go cyberbully him and be super, super mean to him, and Isaac might get really mad at me, and he could sue me.

But if I posted that on Instagram or Facebook, uh, Isaac cannot sue Meta.

And B, sort of what you've been hearing from talking to people in the space about what the implications of this new tool that Anthropic has just launched are, because, uh, they certainly seem worried about it based on everything that I've been reading.

[laughs]

... and try to ensure that we get, you know, whatever we want.

And it's been hugely effective for them, by the way.

Mark Zuckerberg has gotten an insane number of things from Donald Trump, and I'm sure he'll get more, you know, as the years go on.

So I got a lot of pushback from the trust and safety community when I wrote this piece 'cause I was essentially calling them out, just being like, "Hey, like, where are you guys? Like, are you actually gonna get on a microphone anywhere and say, 'Hey, it's really bad what is happening to our industry?'" And what they told me, very justifiably, was like, "We do not have the power that you think we have.

And also, when we do speak up and when people do know our names, we get death threats, and we get hounded to the ends of the earth, and it's really scary, and you're asking us to sacrifice, like, maybe even our lives to, like, speak out in, in favor of these principles.

That's, like, kind of a big ask." So all of that is fair.

Have any of those things worked in Europe yet, or is it just too early to tell?

[laughs]

... is that you can have very wide-ranging political discussions on there.

Uh, you can say really edgy things.

You can say ideas that are sort of fringy and even a little bit dangerous.

And one of the big reasons that you can do that is that the platforms are just confident that if s- if they get sued over this, they can, uh, get the suit tossed rather easily.

So you can imagine, like, a lot of like stuff that people were saying about COVID in the early days, like turned out to be true but was like super edgy at the time.

The platforms just sort of mostly let it happen.

What is Meta's identity right now? Does, does Zuckerberg have a, a coherent strategy, or is he just trying to survive?

It was written in a time when AOL and Usenet existed and were the dominant ways of communicating online, and their goal was to create a competitive marketplace of moderation, where if you wanted your computer to be safe for your kids, you would literally download software and run it locally on your computer, and it would sit in front of CompuServe and, like, filter the internet for you, and that just never happened.It never existed So now I feel like I'm in this place where I'm required to boldly defend a thirty-year-old law whose policy goals were never achieved And I, I don't know why Casey, I know you've been wrestling with this too Like, how, how should I feel about this?

I have complicated feelings too, because I want Section two thirty to exist so that platforms can host political speech, all sorts of speech.

I think that it creates the possibility for platforms that are very, like, rich and vibrant and fun.

At the same time, there's this two thirty case that I paid a lot of attention to as a gay guy about Grindr.

I-- You guys, I'm sure, are familiar with this case.

But basically, there's this horrible ex that's like, "I'm gonna get back at my ex by posting his photos on Grindr, and I'm gonna send everyone his, like, physical address and say, 'Go to this guy's house, and, you know, he's gonna, you know, indulge in your craziest fantasies and, you know, give you drugs.'" And this gets tossed out because of Section two thirty, right? They sue Grindr saying, like, "This is awful.

You gotta do something." And Grindr is like, "Two thirty," and the case goes away.

There's a part of me that buys this, but Casey, I know you think you can pull the two apart.

What his argument, and I think his people's argument, would be is, "Well, now that we're getting uncensored material, now that the First Amendment has primacy, people move to the right because they learn the truth." But the truth is that algorithm incentivizes the misinformation from the right, and he designs it.

A- absolutely.

And like at Platformer, my newsletter, we broke the story a couple years ago that after one of his tweets did not get as much engagement as Joe Biden's during the Super Bowl, he went back to his engineers and he said, "You need to re-engineer this so that my tweets are getting more prominence." And so they did.

And so that is the reason why even if you don't follow the guy and you're using X all the time, you're going to see his views which contain, you know, so many just, you know, various, like, r- right-wing, uh, ideas and conspiracies.

Like, he built the algorithm this way.

And, like, if you're like me and Renee, and you've been covering this stuff for a decade, in 2020 conservatives were holding hearings saying, "Platforms must be ideologically neutral.

You must never suppress any sort of speech.

Mm

Was Highlight cool? I have no memory of Highlight.

So, Highlight was not cool, but Paul Davison is an interesting character.

Like, we all here have interviewed countless entrepreneurs over the years in Silicon Valley.

I would say for me, Paul Davison stands out as one who is, like, a, a visionary in the literal sense of the word.

Like, the man was always having visions, and was very good in telling stories that could sweep up, uh, like, small groups of users and investors.

And so when Highlight was getting ready to relaunch, I went into his, his office and sat down with him, and he explained his vision for a social network where you would just be able to see what anyone around you was doing, and even included their state of motion, so it would, like, tell you if they were, like, on a bike or, like, in a car.

Um, and Paul was able to describe all of this in a way that sounded like not something most people would do, but maybe would become more plausible over time.

Yeah

Like, give us why this story matters, Casey, and what your take is on it.

Yeah, so there are two stories that you mentioned, which I think have some relation, but are probably discussed at least a little bit separately.

So I'll lead with what I just think is the biggest news of the week, full stop, which is that the Pentagon is trying to get Anthropic to renegotiate a contract that it has already signed to enable what the Pentagon calls "all lawful use of Anthropic's technologies." Anthropic, for its part, has said: We're willing to do that, but we want to carve out two exceptions, which is, we want you to agree, Pentagon, not to use any of our technology to conduct mass domestic surveillance, and we also don't want you to use our technology to create autonomous killing machines that have no human in the loop.

And the Pentagon has responded by saying that that is unacceptable, and they've now given Anthropic a deadline of 5:01 p.m. on Friday, and if Anthropic does not comply, they've raised two scary possibilities.

One is that they would designate Anthropic as a supply chain risk, a designation formerly reserved for our adversaries, like Chinese and Russian companies, who we think may be trying to do Americans harms.

This would have economic implications for Anthropic.

Not only would they lose a $200 million contract with the military, but also all other military contractors, like Lockheed and Boeing, would not be able to use Claude in their work.

What is this RSP, as they call it? Why does it matter?

Okay, Casey?

Yeah, I mean, um, uh, you know, as Renee notes, nudification has been a problem going back many years.

In the recent past, though, you couldn't get one of the frontier models to generate a photorealistic image of a naked person.

You would kind of have to hack your way around it and deal with a subpar image.

But nonetheless, there were tons of unscrupulous app makers who were buying ads on Meta and who were buying ads on Apple and Google's Play Stores, getting these apps into the hands of thousands of people who were using them to terrorize especially young women.

Um, and it was a bad problem, a- and it did lead to things like the Take It Down Act, which did pass here in the United States.

But as Renee notes, what has happened on Grok has just taken it to an insane new degree, where you don't have to go scrounge around-

Give, give us an update.

What is that thing?

So last year, I read a blog post by a guy named Andy Matuschak.

I've cited his work a bunch when I write about productivity.

I think he's one of the most interesting thinkers in the world when it comes to this stuff.

Um, but he had this idea called blips, and blips are kind of, like, spiritually similar to this practice that I had been doing of, of saving links.

Like, for him, it was sort of anything that, um, might eventually turn into something more.

What do I mean by something more? Well, maybe it turns into a blog post.

How did that go? What was it like?

If you just put the right guardrails and boundaries and guidelines in place, if you cultivate the vibe of this base, it can actually be good.

good.To me, what is interesting about this is less about who will show up and what will they say on the network, but what can we connect our server to, right? To me, this is the promise of the, the fediverse.

It's not like, could we set up an internet forum where people were nicer to each other and only said like pro-social things about the future of democracy? It's what happens if you're able to link it to some publications that publish news that you think is interesting and link it up to maybe another social network like Threads and see content from people who are posting there but nowhere else.

And then some next generation things, like there are actually like publications that set up their own servers and are sort of publishing directly to this feed and maybe there are some other interactions there.

Th- To me, this is how we actually move away from the internet that we're on.

It is not like, can we get 100 nice people in a room together? I'm sure we could do that.

It is, after we get the 100 nice people in the room, what else can we show them and can it be more interesting than random Instagram reels that were picked for you by an AI? Because like that is the present and the future if nobody else comes up with something better.

Like, you can take your stuff with you when you leave.

But it's not fully AI generated, but there's just a weird, like, paradox there where people seem to be distracted.

I mean, to me, the thread that ties all those together is that AI is a superpower for motivated reasoning.

Like, if there's something you want to believe, like, AI can immediately generate the materials that will help you sell that case to other people.

And, you know, we're all just going to kind of have to, like, upgrade our cognitive hygiene and say, "We now know that there are tools that can get me to sort of believe my own eyes when I shouldn't be doing that." And, um, you know, it's, it's screwing with my mind.

That's why I wrote this piece.

Like, this whole thing is screwing with my mind, and I want all of us to be having a conversation about it.

If anything, anything in the industry changes over the next kind of year?

It, it's a good question.

Casey, I mean, I'm interested in your take here 'cause I, I love what you've written on this stuff.

Thanks.

I mean, you know, people were very concerned about DeepSeek even though at the time, it was not a frontier model, right? What wa- what was interesting about DeepSeek was that it was a little bit less behind than the Ch- Chinese models have been up until this point.

It is also true though that when NBC went out and talked to startup founders in November, what they found was there are a lot of startups that are adopting these free open source Chinese models because they are free.

And so I do think you're seeing that on the rise in Silicon Valley.

At the same time, go to, you know, the Fortune 200 companies.

Ask them who they are using.

Explain what you mean by that, and what did it reveal about how difficult it is to control these chatbots, even when you're the richest man on Earth?

And there's something very confusing to me about right now where if OpenAI wants you to have a relationship with its platform but its platform isn't even kind of about other people, like it wants you to share your funny Sora videos so that you see the funny Sora videos and then more people get online and talk to their chatbots, it's very weird trying to imagine what an AI-mediated social platform is gonna look like and I do kinda have faith in their ability to draw people into it.

Yeah.

I basically do too.

You know, one of the dimensions of this conversation that we haven't discussed yet is that I think a lot of folks who are around our age have a powerful disdain for what they would call AI slop and they're in disbelief that companies keep creating tools to make and share it.

And I'm sympathetic to that view.

It's weird for me to go on Facebook and see Shrimp Jesus and obviously a fake woman who says that she just turned 101 today and made a birthday cake but there's no one here to celebrate and it has, like, 800,000 likes and comments from people wishing her a happy birthday.

It's weird to me that that's, like, so much of our reality and it's easy to get from there to-This is all gonna burn itself out.

Yes, please.