Intelligence Index v5 Rebuilds the Test and Widens Open-Weights Gap to 12
By @sharedot · · 7 pages
- AI Frontier
- Benchmarks
- Open Weights
- Artificial Analysis
Artificial Analysis's Intelligence Index v5 moved 60% of scoring behind private test sets, widening the closed-vs-open gap from 3 to 12 points.
What Happened: The Ruler Changed
Artificial Analysis launched Intelligence Index v5 in late October 2026, its biggest overhaul since the index began, with 60% of the composite now built on private test sets or held-out answers that labs cannot study or train against.
Two new evaluations — Cognition's FrontierCode 2.0 and the 70-task Terminal-Bench-Science — anchor the shift, alongside upgrades to Humanity's Last Exam (replaced by the 1,000-question HLE-Diamond), AA-Omniscience v1.1 and CritPt v2. GDPval-AA v2.1, SciCode and AA-LCR v1.1 were dropped entirely.
The reshuffle left Claude Opus 5.5 on top at 58, ahead of GPT-6 Astra at 57 and GPT-6.1 Sol at 55.
Why It's Surprising: A Reversal for Open Weights
The surprise is the direction of travel for open-weight models. Startup Fortune reports that on the old v4.3 scoring Kimi K3 had closed to within 6 points of the leader, and that Artificial Analysis itself had noted open models had roughly halved the gap over the preceding year.
Trending Topics notes the deepest damage lands on Chinese open labs: DeepSeek V4.1 Flash plunged from 45 to 32, Xiaomi's MiMo-V2.6-Pro lost 16 points to finish at 30, and Meta's Muse Spark Open fell from 39 to 25.
The Evidence: No Model Wins Everywhere
The per-benchmark detail undercuts any single narrative.
OfficeChai reports no model leads all nine evaluations: Opus 5.5 tops FrontierCode 2.0 (46%), HLE-Diamond (68%) and AA-Omniscience v1.1, while GPT-6 Astra leads Terminal-Bench-Science at 63% and the CritPt v2 hard subset at 72%, Gemini 4 Argon tops AutomationBench-AA at 78%, Muse Wonder Full leads GDP.pdf, and — notably — Kimi K3.1 Pro leads the entire field, proprietary models included, on Terminal-Bench 4.0 at 67%.
The new tests are also expensive to run: Trending Topics reports a median of about 59,000 output tokens per task, roughly two-thirds more than before, with Opus 5.5 at max effort costing about $10 per task against roughly $1.27 for GPT-6.1 Sol.
The Stakes for Builders and Buyers
This index is what procurement teams cite when defending seven-figure inference contracts, so a rebuilt methodology changes decisions overnight even though the models themselves did not.
Startup Fortune highlights Meta's Muse Wonder Full as the cautionary marker: it scored 56 on the prior version just two weeks before v5 shipped and now sits at 50 — not because the model changed, but because the ruler did.
There's also a transparency trade-off: because most test data is now secret, Trending Topics points out the results can no longer be independently verified from outside, even as Artificial Analysis argues secrecy is precisely what stops models training on known questions.
What Comes Next
Alongside v5, Artificial Analysis launched Progress Index v1, which places every major language model released since November 2022 on a single continuous capability scale — an attempt to let buyers track how fast the ground shifts rather than resetting the leaderboard every few months, Startup Fortune reports.
Watch whether open-weight labs close the private-test gap as new releases arrive, and whether the community pushback (r/LocalLLaMA has been arguing over the release since it landed, per Startup Fortune) pressures the firm to publish more public companion datasets, as it already did with AA-Omniscience-Public-v1.1 on Hugging Face, according to OfficeChai.
Sources
- startupfortune.com › Claude Opus 5.5 Takes Top Spot as Artificial Analysis Launches Intelligence Index v5 - Startup Fortune
- officechai.com › Artificial Analysis Launches Intelligence Index v5 With Majority Private Datasets, OpenAI Models Gain, Muse Wonder, Kimi Drop
- trendingtopics.eu › Artificial Analysis's New A.I. Index Penalizes Open-Weight Models | Trending Topics