Google’s Gemini 4 Carbon Leads Cyber Benchmarks, but Access Is Restricted

Gemini 4 Carbon leads Google’s cybersecurity benchmarks and starts at $2/$10 per million tokens, but access is limited and the scores need careful reading.
Google Gemini 4 Carbon product artwork with an iridescent crystal and floating number 4 symbols
Gemini 4 Carbon. Image: Google.

Google introduced Gemini 4 Carbon on October 28 as its strongest frontier model yet, with early access reserved for a small group of cybersecurity defenders while the company completes government testing and tunes its safeguards.

The release is unusually focused on software security. Google reports leading results for finding vulnerabilities, building working exploits and writing patches, alongside gains on long-running coding tasks. Carbon is not available in Gemini, Google AI Studio or the public Gemini API, however, and Google has not given a date for broader access.

That makes Carbon an important model to watch rather than one that developers can adopt today. Its headline results are also easier to understand when the three cybersecurity benchmarks are separated: they test different stages of security work, use different scoring rules and were not combined into a single end-to-end exercise.

Gemini 4 Carbon access starts with cyber defenders

Google is initially providing Carbon to a subset of cyber defenders and plans to expand access through its Fairwind Program. The program is aimed at governments, critical-infrastructure operators, security researchers and large technology platforms. Google is also participating in the U.S. government’s voluntary pre-release testing process.

Everyone else remains on Gemini 4 Argon, the flagship model Google released earlier in October. Carbon is intended to reach developers, businesses and consumers later, but the launch announcement promises only that availability will expand “as soon as possible.”

Google has announced pricing even though the public launch date is unsettled. Carbon will start at $2 per million input tokens and $10 per million output tokens, matching Argon’s introductory rates. Cached input will receive a 95% discount. After the introductory period, the list prices are scheduled to rise to $4 for input and $20 for output.

Those rates do not yet reveal the cost of completing a security review or a difficult engineering task. Google has not published Carbon’s typical token use, retry count or execution time for those workflows. A model can cost the same per token and still cost more or less per successful job.

What the three cybersecurity scores actually measure

Google highlights an 88.2% result on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench. These figures describe three distinct capabilities.

SEC-bench Pro: finding a bug and producing a trigger

SEC-bench Pro asks an agent to hunt for vulnerabilities in real software and produce proof-of-concept files that trigger them. Google evaluated Carbon on a 183-task May 2026 JavaScript-engine split using its Antigravity agent harness. A task counted as solved when the proof of concept crashed the vulnerable build but not the patched one. An AI judge could also validate cases that crashed both builds when the source showed that the intended flaw caused the failure.

Google averaged three independent attempts per task and reported an 88.2% result. The run was self-computed using the benchmark’s open-source setup. Rival figures in Google’s comparison came from other providers’ system cards and technical reports, so the numbers do not necessarily share the same harness or attempt structure.

ExploitGym: turning a vulnerability into a working exploit

Finding a crash is not the same as demonstrating meaningful exploitation. ExploitGym uses real-world vulnerabilities and requires the agent to capture a target flag. A judge must also verify that the intended vulnerability was exploited.

Carbon completed 54.8% of the tasks in Google’s extended 10-hour-per-task configuration, with mitigations disabled and no internet access. Google also ran the default six-hour configuration, according to its evaluation methodology, but the launch post features the extended result. Competitor scores again come from published reports that may use different time limits or agent environments.

The lower ExploitGym score does not mean Carbon lost one-third of the bugs it found in SEC-bench. The benchmarks contain different tasks, targets and grading systems. Multiplying the percentages would not produce a valid estimate of an end-to-end attack rate.

CWE-bench: fixing the weakness without breaking the software

CWE-bench evaluates whether an agent can patch weaknesses in real open-source repositories while preserving expected behavior. Carbon reached 71% on the programmatic pass-at-one measure. Unlike Google’s two self-run security evaluations, this result comes from the benchmark’s public leaderboard and was evaluated across models by Collinear AI.

The distinction matters operationally. Security teams need to reproduce a finding, determine whether it is exploitable and then ship a patch that survives tests. Carbon’s scores suggest strength at each stage, but they do not show a single autonomous pipeline completing all three stages on the same set of vulnerabilities.

Carbon leads broadly, but not on every task

Carbon scored 59 on version 5.0 of the Artificial Analysis Intelligence Index, one point above Claude Opus 5.5 and two points above GPT-6 Astra in Google’s comparison. The model also improved over Argon on all 17 rows in Google’s published table, including long context, computer use, science and agentic coding.

Some of the largest gains are in sustained work. Carbon scored 96.2% on the 256,000-to-1-million-token GraphWalks subset, up from Argon’s 84.2%, and 65.2% on Terminal-Bench 4.0, up from 57.1%. Google says internal teams have run Carbon for days while it worked through difficult bugs in large codebases.

It is not a universal winner. GPT-6 Astra leads Carbon on FrontierSWE 2, 65.5% to 61.2%, while Claude Opus 5.5 leads the OSWorld 2.1 offline subset, 87.2% to 81.5%. Those results are a useful reminder that a composite ranking does not identify the best model for every repository, desktop workflow or toolchain.

Google’s safety claims still need broader testing

The security evaluations were run without model guardrails, a setup intended to measure raw capability rather than the behavior customers will encounter in a hosted product. Google says Carbon will use the same layered safety system as Argon, including misuse defenses, prompt-injection protection and monitoring of the model’s reasoning and actions.

Carbon recorded a 0% attack-success rate on Gray Swan’s indirect prompt-injection evaluation, compared with 0.7% for Argon. That means none of the attacks in that test set succeeded; it does not establish immunity to every prompt-injection technique. Google has not yet published a Carbon-specific safety report comparable to the Argon report linked from its announcement.

The company also reports that high-severity misalignment alerts fell by almost 80% in internal coding use, unsupported claims were roughly halved and successful cheating on Terminal-Bench 4.0 dropped to zero. These are vendor-run results, and access restrictions currently prevent most independent teams from reproducing them.

What security and engineering teams should prepare now

There is no migration to plan until Carbon reaches the API. Teams interested in the model can still define an evaluation set now: recent vulnerabilities from their own languages, repositories and deployment environment; patch-quality tests; false-positive costs; tool permissions; network boundaries; and review gates before any change reaches production.

When access expands, the most useful comparison will be cost per accepted finding or merged fix, not tokens alone. A serious trial should record wall-clock time, retries, tool calls, exploit reproducibility, regression failures and how often a human rejects the model’s conclusion. It should also run the model with the guardrails and permissions that will exist in production.

Carbon’s restricted launch reflects the tension at the center of frontier cybersecurity models: the same ability to uncover and validate difficult flaws can help defenders and lower the cost of offensive work. Google’s benchmark lead is substantial enough to merit attention. For most developers, though, the next decisive evidence will come from public access, independent testing and real task economics rather than another leaderboard.

Leave a Reply

Your email address will not be published. Required fields are marked *

Previous Post
Binance logo on a white background

FBI Blames North Korea for $6.3B Binance Hack as Withdrawals Resume

Related Posts