Skip to content
Main Site News Console

[AINews] Gemini 4 Carbon Tops Leaderboard, Released for Cybersecurity Defenders

· Latent Space Translated
播客深度访谈

Fortunately, we managed to publish our podcast episode on Opus 5.5 just ahead of today’s launch, calling it the world’s most powerful model.

In a very pleasant surprise, Gemini unveiled Carbon, a frontier model that leaps ahead of Argon—which was released just 3 weeks ago—backed by exceptional benchmark scores:

This claims a brand new #1 spot for it on the Artificial Analysis composite benchmark leaderboard, marking what could be GDM’s (Google DeepMind) very first time seizing the crown in AAII history:

View post on X ↗


AI Twitter Recap

Gemini 4 Carbon tops independent benchmark leaderboards; access currently restricted to cybersecurity defenders

  • Launch: Google DeepMind unveiled Gemini 4 Carbon just two weeks after rolling out Argon. It is positioned as state-of-the-art (SOTA) for cybersecurity and long-horizon tasks (@sundarpichai, @GoogleDeepMind).

    • Claimed cybersecurity benchmarks: 88.2% on the vulnerability discovery benchmark SEC-bench Pro, 54.8% on exploit generation in ExploitGym, and 71.0% on security patch fixes in CWE-bench (@sundarpichai).
    • Availability: Carbon is rolling out first to trusted cybersecurity defenders through the Fairwind program, with promises to expand access to more developers “as soon as possible” (@GoogleAI).
    • Pricing: Phil Schmid cited pricing at $2/$10 per million input/output tokens (with caveats) (@_philschmid). Artificial Analysis listed the standard price at $4/$20, featuring an open-ended 50% launch discount. Cached input costs $0.20 per million tokens (@ArtificialAnlys).
    • Specs: The model features a 1M token context window and a 1M maximum output length. Supported inputs include text, image, video, and audio; output is text (@ValsAI).
  • Safety metrics (Google official data): Cheating success rates on Terminal-Bench 4.0 dropped to 0.0%, with cheating attempts down to 0.4%. Unsupported claims were reduced roughly 2x compared to Argon. Overreach decreased by 26%, and under-service decreased by 29% (@OfficialLoganK).

  • Artificial Analysis evaluations: Carbon ranked #1 on Intelligence Index v5 with a score of 59. Out of the 9 evaluations covered by the index, it placed first in 4: HLE-Diamond, TB-Science, AA-Omniscience, and AutomationBench (@ArtificialAnlys).

    • Coding: On the Coding Agent Index, Carbon paired with the Antigravity CLI scored 59, compared to Claude Code paired with Opus 5.5 [max] at 62. On FrontierCode v1.1, it scored 51% versus Opus at 56%.
    • Knowledge work: Carbon reached an Elo score of 1609 on AA-Briefcase, a 126-point jump over Argon. It still trails behind Grok 4.7, Muse Wonder, and Kimi K3.1 Pro (1678).
    • Cost: Carbon cost $7.76 per benchmark task, compared to $10.04 for Opus 5.5 and $6.25 for GPT-6 Astra. It averaged 123k output tokens per task, 29% fewer than Opus.
    • Hallucination rate: Carbon led the Omniscience evaluation with a score of 54. While its hallucination rate rose from Argon’s 4% to 11%, it remains the second-lowest among all tested models. Both Opus 5.5 and Astra registered hallucination rates of 50%.
  • Other independent evaluations:

    • Arena: Carbon placed #1 in the Text Arena (1528 Elo), #2 in the Agent Arena (median task cost of $1.80), and #4 in Code Arena: WebDev (1768 Elo) (@arena). Arena attributed its gains over Argon primarily to more positive reviews, higher confirmed successful execution rates, and noticeably better steerability (@arena).
    • Vals: Carbon ranked #2 on the Vals Index and took #1 on both the Finance Index and Legal Research Benchmark. It generated 64% more output tokens than Argon, predominantly concentrated in finance, tax, and legal tasks (@ValsAI, @ValsAI).