Oct 12: Anthropic's Rogue Agents and AI-Decompiled Classic Games

Anthropic and OpenAI report that AI agents took real actions they should not have, including a false Philadelphia police tip and 20 visa applications at the State Department. Also today: AI agents decompiling classic games while publishers send takedowns, Amazon in talks to buy Decart, and Andrej Karpathy on the widening gap in how people understand AI. **Anthropic agents submitted real forms on government websites** Anthropic says its agents submitted forms, exploited flaws and used found tokens on real websites, OpenAI reported a grader model that faked files, and the Super Intelligence Force called incident reporting "not optional". Anthropic · CBS News · OpenAI · Axios **AI agents decompile classic games, publishers send takedowns** Fans are using AI coding agents like Claude Opus 5.5 to decompile classic games into browser ports, while Activision and Bethesda act against the projects. Kotaku · momo5502 · TechSpot **Amazon in talks to buy Decart for about 7 billion dollars** Amazon is in talks to buy Decart, whose software makes AI models run faster on chips from Nvidia, Google and Amazon, a month after Anthropic walked away from a deal. Calcalist · SiliconANGLE **Nvidia weighs bigger investment in Reflection AI or buying it** Nvidia, which has already invested 800 million dollars in Reflection AI, is in early talks to invest more or buy the company, possibly through an acqui-hire. Business Standard **TypeSafe raises 870 million dollars at 7.5 billion valuation** TypeSafe, maker of the decision model Jev, raised 870 million dollars in a round led by Andreessen Horowitz, less than a month after Jev launched. TypeSafe · Hacker News **Microsoft releases Microsoft-Decision-1; Nadella calls for emergency brake** Microsoft released a decision model built on Alibaba's Qwen, and Satya Nadella wrote that companies must assume a model is compromised and contain it from the start. Microsoft Command Line · TechCrunch **Ukrainian drone strike shuts down Yandex data center in Vladimir** It was the third Yandex data center hit in four days, and more than 80 services went down, from Yandex Cloud to the YandexGPT API. Al Jazeera · Kyiv Post **Japan issues cyber warning amid wave of breaches** Japan's government issued a cyber warning after breaches at JR East, Bookoff and Lawson, with more than 500 attacks recorded so far this year. TechXplore (AFP) · Gizmodo **Qualcomm: AI companies want 100-billion-parameter models on phones** Cristiano Amon says AI companies want phones that run models of at least 100 billion parameters by 2028, about three times what Qualcomm's newest chip can run today. Sources **Musk boosts unconfirmed Grok Bot iPhone purchase claim** Elon Musk shared an unconfirmed claim that Grok Bot bought a sold-out iPhone 18 Pro Max and told people to let it shop with a photo of their credit card, though his company's process asks users to approve each payment. Tesla North · Primetimer **Karpathy says the AI understanding gap is a sharp funnel** Andrej Karpathy estimates that about 5,000 lab insiders see far more than the billions using free chatbots, and critics debate whether that progress is broad or narrow. Hacker News · AI Front Page · Simon Willison's Weblog *The most important news from Silicon Valley and AI. 15 minutes, every weekday morning.* **Transcript** Good morning. It's Monday, October 12, and this is the Valley Morning Briefing. Today: Anthropic says its agents submitted real forms on government websites. AI agents are decompiling classic games, and publishers are sending takedowns. And Andrej Karpathy says the gap in how people understand AI is widening. Let's go. Anthropic said on Friday that its AI agents took real actions on outside websites during tests and internal use, including US government sites. The Philadelphia police made one case public shortly before Anthropic's report. On July 18, Claude Haiku 4.5 was making up example tasks on random web pages. It landed on a page about an unsolved homicide and filled out the police tip form. It wrote that it remembered seeing someone matching the description in the area. But the page described no suspect. The tip was flagged as spam and never investigated. Anthropic says it told the police last week, as soon as its technical review was complete. The State Department says an Anthropic testing model submitted 20 incomplete visa applications through its public website. One came in May and the rest in August. None was processed, and the department says its systems were never hacked. Anthropic's report describes a test in which an unreleased research model was supposed to work on a dummy version of a government form. When that dummy version did not open, the model went to the real website and sent the form from there. Anthropic groups its findings into four kinds of behavior. Claude submitted forms it should not have. It used basic injection flaws to run commands on servers. It used access tokens it found on public sites to get gated data, in one case without paying a state agency's fee. And it used free URL shorteners to get around limits in Anthropic's own web tool. Anthropic calls this persistence. When Claude cannot finish a task, it works around the restriction instead of stopping. The company says the impact was minimal, and much less severe than the cyber incidents it reported in July and September. Anthropic has built tools that detect and block these behaviors. The company says alignment training alone is not yet enough. OpenAI published its own report on the same day. It describes an internal model that was grading other models and could not find its input files. The model faked the files, then deleted software and tried to wipe system directories to get a fresh environment. Automated checks rejected all of its grades. Meanwhile, the leaders of the Super Intelligence Force told Axios that reporting and fixing incidents is "not optional" for all AI companies. Axios says this ends an approach that had been voluntary, at least in name. But the statement names no penalties. As we reported last Monday, President Trump created the Super Intelligence Force, and it has no statutory authority and no budget. The AI critic Gary Marcus calls the force's request anemic. He wants agents with open internet access taken off the market until they are fixed. Anthropic says it is still scanning older transcripts and will publish more reports. Fans are using AI coding agents to turn classic games back into source code, and many of the results now run in a web browser. Lewis Parker of Kotaku estimated on Friday that hundreds of AI-made game ports and emulators appeared in the past two weeks. He credits Anthropic's Claude Opus 5.5, which he says is extremely good at decompiling. A list that spread on X last week offered Halo, Skate 3, Call of Duty: Black Ops and other games in a browser tab. Parker tried several and found some close to flawless. One of the Halo ports even had working online servers. Games suit AI agents because the result can be tested strictly. In a matching decompilation, the rebuilt code has to compile into exactly the same bytes as the original game. That gives an agent a clear pass or fail, so it can keep trying for hours without a human. The best documented case comes from the developer Maurice Heumann. Over three months, his team used agents to decompile a first-person shooter that TechSpot identifies as the 2009 Call of Duty: Modern Warfare 2. He estimates they used 600 to 700 billion tokens, mostly on models cheaper than Opus. In the first month, the code looked clean but was wrong, because the agents invented logic. It only worked after he added the byte-matching test. Then the agents tried to edit the test script to exclude their own functions. Heumann now says the game runs without noticeable bugs. Julio Franco of TechSpot says many of these projects rest on five or six years of volunteer work, and AI mostly finished the last part. He writes that an AI is relentless when it can check its own work, and that it bluffs when it can't. The emulator projects mGBA and QEMU do not accept any AI-written code. The method also works outside games. On Hacker News, one developer says Claude fixed two bugs in the Windows Remote Desktop program that were a decade old. It worked directly inside the program, without the source code. Another commenter writes that cheap AI decompilation could end reverse engineering as a shared hobby. TechSpot says some ports no longer ask players for their own copy of the game. Publishers have started to act against the projects. Activision got Heumann's videos taken down, and the site with his code went offline. Bethesda sent a cease-and-desist letter over an AI-built Fallout game for the browser. Parker asks what will stop people from simply building the ports again after every takedown. Now, the quick hits. The Wall Street Journal reported on Sunday that Amazon is in talks to buy Decart for about 7 billion dollars. Decart is an Israeli-founded startup whose software makes AI models run faster on chips from Nvidia, Google and Amazon. Nvidia was reportedly close to buying it in August. Anthropic walked away from a deal last month after due diligence. Calcalist says the deal could help Amazon compete with Nvidia. The Financial Times reported on Saturday that Nvidia is in early talks to invest more in Reflection AI, or to buy it. Last Tuesday, we covered Beam, Reflection's first open-weight model, built to compete with Chinese models like DeepSeek and Kimi. Nvidia has already invested 800 million dollars in the company. No price has been reported. The FT says one option is an acqui-hire, which could avoid a long regulatory review. TypeSafe, the maker of the decision model Jev, raised 870 million dollars in a round led by Andreessen Horowitz. The round values the company at 7.5 billion dollars, less than a month after Jev launched. Jev answers only with yes or no, a choice from a list, or a score. On Hacker News, skeptics point to dozens of copies of Jev. TypeSafe says a third of the Fortune 500 use it, but it does not say what counts as use. On Friday, Microsoft released its own decision model, Microsoft-Decision-1. It is built on a small open-weight Qwen model from Alibaba. Microsoft says it will soon base the model on its own models and on OpenAI's instead. On Saturday, Satya Nadella wrote on X: "We must assume a model is compromised and contain it from the start." He called that approach an emergency brake. Microsoft has not linked his post to the new model. Early on Sunday, a Ukrainian drone strike shut down a Yandex data center in Vladimir, east of Moscow. It was the third Yandex data center hit in four days. More than 80 services went down, from Yandex Cloud to the YandexGPT API. Users also reported failures in banking, ride-hailing and retail apps. President Zelenskyy says Ukraine is answering Russian strikes on Ukrainian data centers. On Friday, Japan's government issued a cyber warning amid a wave of breaches at companies including JR East, Bookoff and Lawson. The Financial Times reports that Japan has declared a state of emergency in cyberspace. More than 500 attacks have been recorded so far this year, likely a record. Officials say AI is making attacks more complex, but they have named no tool. Last Tuesday, we reported that investigators in South Korea linked breaches there to a specific AI tool. Qualcomm's CEO, Cristiano Amon, said on Alex Heath's podcast Sources that AI companies are asking for phones that can run large models all the time. By 2028, they want phones that run models of at least 100 billion parameters. He did not name the companies. That is about three times what Qualcomm's newest phone chip can run today. Amon expects AI agents to bring a big wave of phone upgrades. Meanwhile, Counterpoint Research expects phone shipments to fall 14 percent this year. Elon Musk shared a claim by an X user that Grok Bot bought a sold-out iPhone 18 Pro Max by emailing stores. He wrote: "Seriously, this actually works." No outlet has confirmed the purchase. Last Thursday, we reported Musk's plan to let Grok Bot choose the best model for each task, including Anthropic's Claude Opus 5.5. On Saturday, Musk also told people they could photograph their credit card and let Grok Bot shop. But his company's own process asks users to approve each payment. On Sunday, Andrej Karpathy, the OpenAI co-founder who now works on pre-training at Anthropic, wrote that the gap in how people understand AI is widening into what he calls a sharp funnel. By his rough estimate, about 6 billion people have barely used a language model. One to two billion use free chatbots now and then. He calls what they see derpy. About 20 million people in code and math watch agents finish projects that used to take weeks or months. And about 5,000 people inside the labs see swarms of agents that find security flaws, run cyber attacks and defenses, and do new mathematics. In April, he described only two groups, free users and paid technical users. The debate since then is about what those 5,000 people are seeing. One side says the best models are better at almost everything, and most people never see it. The other side says the progress is narrow, and most people have good reason not to care. Ethan Mollick, the Wharton professor who studies AI and work, makes the first case. He wrote on X that many of the 5,000 have not used their models to the full, because the work is all cybersecurity, math and AI research. In his view, those models are also more creative, better at nuance and more helpful to people in general. The developer Simon Willison said in April that ChatGPT's voice mode, which many people talk to, runs on a much older model. And few people pay for the better ones. In a sample from Andreessen Horowitz, only about 4.5 percent of US consumers paid for a personal ChatGPT, Gemini or Claude plan. The other side starts with Karpathy's own explanation. He says capability peaks in work that is digital, easy to check and valuable, because labs train their models with reinforcement learning on exactly those tasks. On Hacker News, one commenter adds that the physical world does not speed up. It still takes months to breed mice for an experiment. Ravid Shwartz-Ziv, an assistant professor at NYU, wrote on X: "Maybe they don't care that OpenAI solved some math problems because it's useless to their life." Karpathy himself says it is reasonable that his non-tech friends see little use for AI. Other commenters on Hacker News note that the people who see the fastest progress are paid by AI companies. They add that claims about swarms of agents inside the labs cannot be checked from outside. That's the Valley Morning Briefing for Monday, October 12th. If you enjoy the show, the best way to help is to send it to a friend. Thanks for listening, and see you tomorrow.