Altman's "terrible" ending shows why scenario talk isn't AI safety

What do you call a CEO who lays out three possible endings to the race he is running, rejects the worst one, and then explains why his company can't afford to do much about the other two? We'd call it honesty, and also a warning label.

Altman's three endings are a menu he can't order from

According to The Information, Sam Altman described three endgames for the AI race at an internal meeting. The first is an international body modeled on the IAEA, with power to inspect systems and demand audits. The second is the "Demis Plan," in which the leading labs merge or work under one roof. The third is the "Eliezer Plan": one developer gets self-improving AI first and uses that advantage to deprive every rival of compute, which in the phrase that went around would melt everyone's GPUs. Altman reportedly called it, in the words of 36Kr's account, an "utterly terrible" outcome.

The account is based on anonymous sources, so treat the details with care. But the detail that matters most isn't the scary one. As Martin Cid reports, two of the three ideas echo a 2023 OpenAI post, and none was ever implemented. Altman also reportedly said the first two options require deep coordination, and that OpenAI can't cede ground now, with it and Anthropic heading toward public listings.

The "terrible" ending is the default, not the outlier

Read that again. The good endings require cooperation. The bad one requires only that somebody win. In a race where nobody is willing to be the one who slows down, we think the winner-takes-all path isn't a remote tail risk; it is where the incentives lead unless someone pays to change them. On our reading, Altman's reported position amounts to this: he is against the outcome, but also against giving up ground that could help avoid it.

That isn't hypocrisy so much as a structural trap, and he's hardly alone in it. Earlier this fall, 01net recalled, Altman said he agreed with Anthropic's Dario Amodei's call to slow down but didn't want to be the only one to ease off. Everyone is willing to brake, provided someone else brakes first.

Scenarios without commitments are PR

Here's our position: describing futures is cheap, and binding yourself is expensive. The reported evidence of binding is thin. The self-regulatory body that OpenAI, Google and Anthropic are reportedly building, the Standards Authority for Frontier AI, is described as lacking enforcement power, and the labs reportedly still disagree about who should test new models. A cartel complaint in the US already argues that safety coordination among labs is collusion. Meanwhile Meta's and Nvidia's chiefs reportedly argue such rules mostly protect incumbents.

Some of those objections deserve a hearing. A single merged super-lab, the "Demis Plan," would concentrate enormous power, and an international inspectorate needs Washington and Beijing to let inspectors into their most sensitive labs. Forbes, as relayed in the same report, notes that even a US proposal for the two countries to tell each other about dangerous AI incidents is still only a draft, not an agreement. And in his Politico interview, as 01net relays it, Altman himself pushed back on the idea that a single lab in San Francisco should hold this technology, which cuts against the merger idea too.

But "every fix has problems" is exactly why commitments matter. A company serious about the terrible ending would put something costly on the table: a pause trigger, an outside auditor with real access, a compute cap it accepts first. A presentation to staff doesn't qualify.

Judge labs by what they would give up

We shouldn't be cynical about the candor. It's useful that an insider says out loud that alignment is unsolved; Lefilia reports that Altman holds sandboxes and monitoring are only stopgaps. But candor about the risk plus refusal to be first to constrain yourself is a description of the problem, not a solution.

Our verdict: judge labs by what they would give up, not by which scenarios they can articulate. Until one of them accepts a limit that hurts its own position before an IPO, the three endings are a conversation starter, and the third remains the default setting.