Gemini 4 Carbon's crown means little until defenders get it first

When a model is so good at finding and exploiting bugs that its maker won't let you use it, is that a product or a press release? Google's Gemini 4 Carbon is both, and which one wins depends on what happens in the next few weeks.

Here is the short version. Google launched Carbon on Wednesday, as The Verge reports, touting "significant gains" in coding, agentic tasks and cybersecurity, plus the top spot on the Artificial Analysis Intelligence Index (v5.0), with Claude Opus 5.5 and GPT-6 Astra behind it. It is available first to a select group of cyber defense companies through the Fairwind Program, with broader access to come. Introductory pricing is $2 per million input tokens and $10 per million output tokens.

The crown is thinner than the headline

Google cites a score of 59 on the index. Ars Technica puts that in perspective: every frontier model lands in the same neighborhood on these tests, and the gap between Carbon and the competition is just a few points. It also notes that Carbon beats Argon by slim margins in every test Google has given it, and that for all anyone knows it could be a post-training upgrade rather than a new model.

We think that's the right level of skepticism. A leaderboard lead of a few points, announced by the company that benefits from it, is marketing until someone else reproduces it. The cybersecurity numbers deserve the same treatment: 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71% on CWE-bench are Google's figures, and the claim that Carbon performed more than 10X better at generating high-quality patches is measured against an unnamed "another lab's" model. Unnamed rivals make for unfalsifiable bragging.

The access order is the real story

The more interesting choice is how Carbon ships. Google says it is following the US government's voluntary review process and starting with defenders. That's the right instinct. A model that scores well at both finding vulnerabilities and turning them into exploits is dual-use by construction. Handing it to defenders first only helps if they get a real head start, and that is a promise about timing, not capability.

And the timing is vague. Google makes no promises about a timeline for wide release. Ars suggests that if it goes like Argon, Carbon could be available in a couple of weeks, but that's an inference, not a commitment. Two weeks is a long time in security. If attackers have comparable tools from elsewhere, or Carbon reaches the public before defenders have patched what it finds, the "defenders first" framing becomes a nice story about a short lead.

There is also a quieter issue: a closed preview means nobody outside the select group can check any of this. Fairwind partners presumably have the incentive and the skill to test it hard. We would love to see them, or independent labs, publish what they find, including the failures.

What a skeptical reader should do

Credit where due: the prompt-injection result is notable if it holds. Ars reports Carbon resisted every attempted attack in the Grey Swan evaluation, against 0.7 percent for Argon and 8.5 percent for GPT-6 Astra. That matters more for agents that run for days than a small index lead does. But it's still one evaluation, reported on terms Google chose to publish.

Our verdict: treat the index crown as a footnote and the cybersecurity claims as a hypothesis. If you work in security, ask for the evaluation details and a trial through a Fairwind partner rather than trusting the chart. If you're a developer, don't rearchitect anything around Carbon yet; the introductory price is attractive, but The Express Tribune reports it is set to rise to $4 and $20 per million tokens. The benchmark crown will probably be forgotten at the next release. What will last is whether defenders actually patched things before everyone else got the same capability.