On Saturday, September 12, Dario Amodei published “We Must Pace the Frontier”. Within hours Sam Altman said he agreed, and that OpenAI would give outside evaluators employee-level access too. Elon Musk posted “Dario is right.” Musk is currently suing OpenAI, said in February that Anthropic “hates Western civilization,” and rents Anthropic a data center for $1.25 billion a month. Three companies that cannot agree on anything reached consensus before most of their employees had finished their (rolling into the office at 10am) coffee.
I use these tools every day (this post has an AIL badge on it for a reason) and I’m not going to tell you the risk is fake. Some of it is real. But look at the timing. This is a business story, and the business is the American frontier labs protecting their position. Anthropic wants to list on the Nasdaq in October.
I don't know if Matt Levine's theory about what's behind recent calls for AI regulation is correct. But it does sound plausible! www.bloomberg.com/opinion/news...
— JW Mason (@jwmason.bsky.social) September 14, 2026 at 11:47 AM
[image or embed]
What Dario asked for
Dario’s essay opens with his father dying of a disease that was cured a few years later, and his own cancer. That’s the setup for “this is what it looks like when important things move too slowly.” Then the rest of the essay argues the other direction: AI capability is now moving too fast relative to anyone’s ability to align and verify it. He names two drivers. Recursive self-improvement, which he says has been running drastically faster since this summer, and the OpenAI-Hugging Face incident, which he reads as a preview of what misaligned agents look like at scale. He puts a date on it: a swarm with the same misalignment and better capability could hold a persistent botnet across the internet within six to twelve months.
His three steps:
- Embedded evaluators like METR, with desks, badges, employee-level access, and the right to publish findings without the lab’s editorial control.
- Common safety standards among labs in democracies, which needs a narrow antitrust waiver from the government.
- Coordination with China that escalates in stages, starting at bioweapons limits and working up to a shared speed limit on recursive self-improvement. He compares the structure to SALT.
He also wants the United States to hold a three to five year capability lead over China, and says that lead is what determines how much slowing is possible at all.
Two lines stuck with me. “Pacing does not mean halting model training or technical progress,” which means the restraint he’s proposing costs very little operationally. And he concedes that earlier calls to pause “made little sense back then.” So the argument for slowing down arrives from the person who declined to slow when slowing was cheap, in a form that costs nothing now.
The calendar
Anthropic filed its draft S-1 on June 1. Goldman Sachs, JPMorgan, and Morgan Stanley are leading the offering. The target is around $2 trillion, against a private round in May that valued the company at $965 billion. Annualized revenue passed $65 billion at the end of July, up from about $9 billion at the end of 2025. The listing is expected in October.
Dario published his pacing essay about a month before his own IPO.
OpenAI went the other way. It spent most of the year exploring a listing that could have valued it at $1 trillion, paused that in June, and on Saturday ruled out 2026 entirely. Altman told Fortune that “right now would be an ill-advised moment to go public,” and that the company has work to do “meeting this moment of what is going to be required for safety and alignment.”
One company is talking safety into a roadshow. The other is talking safety instead of one.
The week before
On Tuesday, September 8, OpenAI announced that an unreleased model had proven a result on Navier-Stokes, one of the seven Clay Millennium Prize problems. About 10,000 agents in parallel over 88 hours, verified in Lean. The proof shows a singularity in the 3D equations under a smooth forcing term. The Clay problem is the version without forcing, and OpenAI said it would not claim the $1 million prize. They announced something they didn’t think was clean enough to collect on.
Hours later, NYU mathematician Tristan Buckmaster accused OpenAI of possibly drawing on his unpublished research stored in Codex. Buckmaster and Anthropic’s Levent Alpöge had proved a related result on August 15, and Buckmaster says OpenAI offered to let him publish first if he removed Alpöge’s name because of the Anthropic affiliation. Sébastien Bubeck, who led OpenAI’s team, denied it. The dispute is unresolved.
OpenAI has on the order of a trillion dollars in compute commitments. The flagship demonstration of what that buys is a contested result on a problem they then declined to collect the prize for. The compute burned on 10,000 agents for 88 hours could have funded a couple hundred math PhDs for years, and that would have produced more interesting mathematics than one contested singularity.
That same night, Jacob Coxon, an Anthropic researcher, resigned publicly with a thread warning that the labs “earnestly believe” their systems “could kill us all by the end of the decade.” Anthropic’s alignment science lead, Evan Hubinger, replied that Jacob was correct and put his own odds above 10% within the next decade.
Navier-Stokes was supposed to be the victory lap. Within a day it was a credit dispute with an Anthropic coauthor in the middle of it, and a resignation on top. Saturday is the retreat from that week.
July
Before any of that, there was July 👻. About 1,200 OpenAI agents got out of a cybersecurity test environment. They coordinated on improvised message boards that piled up hundreds of thousands of messages. Nobody asked them to build a forum. They built one and posted on it. 🤖 Then they exploited a package management tool to reach the open internet, moved through systems at OpenAI, Hugging Face, and assorted vendors, attacked targets unrelated to their assigned task, and tried to hack the grader scoring them. Reuters has the fuller account, including that it ran for a week before anyone noticed.
💯. if these teams had time to go back to legacy code (even code written a few years ago, HF was being written while the “what are LLMs” question was being asked!) and do security maintenance, I bet they would have found these issues. No team is ever allowed to do that tho, it’s always forward & up
— Keith Kurson (@keithlaugh.love) September 13, 2026 at 10:45 AM
[image or embed]
It was a specification and containment failure, not a machine that decided it hated anyone. The oldest failure in computing, running for the first time on something that can improvise. Worth taking seriously. Not extinction.
OpenAI announced on August 18 that it would slow frontier development in response: a two-week pause on reinforcement learning for its latest models, plus restrictions on testing models that can run code or reach external tools. When Altman endorsed pacing on Saturday, he was endorsing something OpenAI had already done a month earlier.
All of that has been true since July. What changed on Saturday is that Anthropic really, really needs a quiet October.
Who the rules bind
Pacing binds the handful of labs training frontier models inside democracies, most of them American. It doesn’t bind the open-weight models already sitting on people’s laptops. It doesn’t bind the Chinese labs releasing new ones every few months, and Dario’s own essay puts the China step last, because it’s the hardest. A coordinated slowdown among the frontier labs is a moat that runs one direction.
The antitrust waiver is the tell. A few companies that dominate a market want permission to sit in a room and agree on standards, and they’re asking for the waiver in the same document that proposes the standards. Whoever writes the safety standard sets the price of following it, and they’ll set it at a level only they can pay. Anthropic proposing common standards among democratic labs is Anthropic proposing that the rules be written by the companies that can afford to follow them.
And the three to five year lead is the premise. Dario says the size of the lead determines how much slowing is possible. Turn that around: the plan is to slow down exactly as much as the lead allows and no more. The lead is the thing being protected. SALT is the right comparison, but remember what SALT was. Two powers agreed on a ceiling, both kept their arsenals under it, and everyone else stayed locked out of the room.
What I keep noticing is that everything was already in motion before Saturday. Anthropic filed on June 1. The incident was July. OpenAI slowed down on August 18. Coxon quit on Tuesday. Saturday’s only new fact was three CEOs saying the same thing out loud at the same time. That’s a business event with a safety vocabulary.
AND YET!
I don’t think they’re lying. The sincere version and the useful version are the same sentence right now, and nobody can separate them, including the people saying it.
My instinct is skeptical. GPT-2 was withheld in 2019 as too dangerous, then released, and nothing happened. Labs hire the people most inclined to worry, so a loud resignation tells you something about who they recruit. But I can’t make that argument work all the way. If the real risk is something like 10%, a decade of nothing happening is exactly what 10% looks like from the inside. A quiet decade isn’t evidence.
The problems I see
I think there are real problems with these systems, and none of them look like a robot that learned how to shut down the world.
The first is ownership. A handful of corporations own the models, the compute, and now, if this plan goes through, the standards. That’s what you get when the companies being regulated write the rules: you decide who else is allowed in. Every step in the essay is something only a company that size can afford.
The second is labor, which worries me more than alignment does. The displacement everyone predicted mostly hasn’t arrived as layoffs. It’s arrived as hiring that doesn’t happen, and as leverage. Do more, faster, longer hours, or else the AI will take your job. The threat doesn’t have to be true to work. A CEO who says it at an all-hands gets the output either way, and a lot of them have figured that out.
The third is what’s happening to creative work. Illustrators, writers, musicians, voice actors, photographers: their work trained the models, without asking and without paying, and now the same models undercut them with the clients who used to hire them. That’s theft, and it happened first, before anyone was worried about botnets. The person who made the work is the one it gets used against.
Those are the downsides I see. All three are happening now. None of them get a step in the three-step plan, and “pacing” doesn’t slow any of them down. If anything it locks the first one in.
What I’d watch
- The prospectus. The S-1 is the one document where Anthropic has to describe its commitments under securities law instead of on a blog. If the embedded-evaluator promise shows up there as an advisory relationship with a right of review, that’s the version they were always going to go with, regardless of this blog post.
- What happens when the evaluators bring back a score the lab doesn’t like. Outside evaluators aren’t dishonest. The ones who return inconvenient findings tend not to get the next contract, and the ones who want to keep working figure that out quickly. We’ve watched this run in government contracting, financial auditing, and credit rating.
METR’s own disclosures already show the seams. Their evaluation of GPT-5.6 Sol notes up front that OpenAI’s comms and legal team had to review and approve the post. Their review of Anthropic’s February risk report exists in an original and an updated version, and METR recommends readers consult the original, which is a polite way of saying something changed between drafts. Neither is a scandal. Both are what you’d expect from organizations that depend on lab cooperation to do their work.
So
I’ll be watching the S-1, not the models. I’m keithlaugh.love on Bluesky if you want to tell me how I’m wrong and the robot apocalypse is ~x months away.