Claude Opus 5 · 官方评测披露

Claude Opus 5 当前收录 46 项测试的官方记录。已完成列明材料的核查。

模型发布 · Claude Opus

官方材料与核查范围

The current Opus 5 announcement and 198-page System Card are reviewed, and the later Opus 5.5 card source is complete using Anscombe’s same-URL 230-page full-card review (including all 126 visual pages). Launch-day bytes remain unverified. 核对时间 。

当前资料不足以还原首发。 没有发布当日历史快照。

  • Introducing Claude Opus 5 · 主文 · Current saved page: 本地留存副本 and 本地留存副本; page states Published Jul 24, 2026. The cited 本地留存副本 is absent from the local raw directory; current page revision date is unknown.
    公布 · 采集 · 已核查 已补完当前发布主文全文、全部引言与脚注,实际查看全部14个figure及正文/品牌图像。核读与图片清单见本地留存副本。未估读无数值曲线,客户私有评测或缺方法的引述不自动作为普通能力成绩。
  • Claude Opus 5 System Card · 附件 · Current saved PDF/TXT: 本地留存副本 and current-opus-5-system-card.txt; cover dated 2026-07-24; p.2 changelog records an update on 2026-08-19; later revision date, if any, unknown.
    公布 未确认 · 采集 · 已核查 All 198 PDF/TXT page segments read; all 116 image/vector pages rendered and visually inspected; no unread pages or high-information figures remain.
  • Claude Opus 5.5 System Card (Opus 5 OSWorld 后续对照) · 附件 · Current saved PDF/TXT: 本地留存副本 and current-opus-5-5-system-card.txt; cover dated 2026-09-22; 230 pages and all 126 visual pages fully reviewed in 本地留存副本; current revision date unknown.
    公布 未确认 · 采集 · 已核查 The current saved 230-page PDF/TXT was fully read page by page and all 126 image/vector pages were rendered and visually inspected; the completed review is documented in 本地留存副本 and its source-review patch.

配置

  • Adaptive thinking · max effort (card-wide default) · System Card §8.1 default: adaptive thinking at max effort, default sampling settings (temperature/top_p), averaged over five trials unless otherwise noted; context varies by evaluation and does not exceed 1M tokens. max
    未知项 Per-benchmark exceptions and harness details are recorded on the relevant configurations/results.
  • ARC-AGI-3 High effort · System Card table marks the 30.2% ARC-AGI-3 result as high effort. high
    未知项 Version and harness details not stated in the table.
  • HLE · thinking auto, no tools · Reasoning-only configuration; thinking=auto; 1M total-token cap across contexts; no context compaction; Opus 4.6 model grader. max
    未知项 The headline result does not identify a low-to-max effort setting.
  • HLE · thinking auto, with tools · Web search, web fetch, programmatic tool calling, and code execution; thinking=auto; 1M total-token cap across contexts; no context compaction; Opus 4.6 grader; HLE-specific URL blocklist and transcript contamination review. max
    未知项 The headline result does not identify a low-to-max effort setting.
  • OSWorld 2.0 更新 harness · System Card 后续披露的 2026-09-10 task/assets 版本,108 题,保留全部截图并启用 server-side context management。
    未知项 System Card 修订日期未公布;与 launch table 的原 OSWorld 2.0 值不可直接比较。
  • FrontierBench v0.1 · Harbor evaluation · The Table 8.1.A Harbor result; the card-wide default is adaptive thinking at max effort, default sampling, and five trials unless noted. Harness-specific details for the Harbor row are not stated. max
    未知项 Do not merge with the distinct Anthropic mini-SWE-agent/GKE run in §8.5.
  • FrontierBench v0.1 · mini-SWE-agent/GKE · Low effort · Anthropic-run mini-SWE-agent harness on GKE; mean reward over five attempts for each of 74 unique tasks; production safety classifiers were enabled. Refusals triggered fallback to Opus 4.8 in 4% of total trials. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierBench v0.1 · mini-SWE-agent/GKE · High effort · Anthropic-run mini-SWE-agent harness on GKE; mean reward over five attempts for each of 74 unique tasks; production safety classifiers were enabled. Refusals triggered fallback to Opus 4.8 in 4% of total trials. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierBench v0.1 · mini-SWE-agent/GKE · Xhigh effort · Anthropic-run mini-SWE-agent harness on GKE; mean reward over five attempts for each of 74 unique tasks; production safety classifiers were enabled. Refusals triggered fallback to Opus 4.8 in 4% of total trials. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierBench v0.1 · mini-SWE-agent/GKE · Max effort · Anthropic-run mini-SWE-agent harness on GKE; mean reward over five attempts for each of 74 unique tasks; production safety classifiers were enabled. Refusals triggered fallback to Opus 4.8 in 4% of total trials. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSWE v1.1 · Low effort · 113 long-horizon software-engineering tasks; Figure 8.3.A effort/cost sweep. The figure does not state effort-specific trial counts. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSWE v1.1 · Medium effort · 113 long-horizon software-engineering tasks; Figure 8.3.A effort/cost sweep. The figure does not state effort-specific trial counts. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSWE v1.1 · High effort · 113 long-horizon software-engineering tasks; Figure 8.3.A effort/cost sweep. The figure does not state effort-specific trial counts. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSWE v1.1 · Xhigh effort · 113 long-horizon software-engineering tasks; Figure 8.3.A effort/cost sweep. The figure does not state effort-specific trial counts. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSWE v1.1 · Max effort · 113 long-horizon software-engineering tasks; Figure 8.3.A effort/cost sweep. The figure does not state effort-specific trial counts. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Main · Low effort · Cognition-run FrontierCode evaluation; 150 tasks; overall composite score reported as mean@5. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Main · Medium effort · Cognition-run FrontierCode evaluation; 150 tasks; overall composite score reported as mean@5. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Main · High effort · Cognition-run FrontierCode evaluation; 150 tasks; overall composite score reported as mean@5. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Main · Xhigh effort · Cognition-run FrontierCode evaluation; 150 tasks; overall composite score reported as mean@5. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Main · Max effort · Cognition-run FrontierCode evaluation; 150 tasks; overall composite score reported as mean@5. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Extended · Low effort · Cognition-run FrontierCode evaluation; extended set; overall composite score reported as mean@5. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Extended · Medium effort · Cognition-run FrontierCode evaluation; extended set; overall composite score reported as mean@5. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Extended · High effort · Cognition-run FrontierCode evaluation; extended set; overall composite score reported as mean@5. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Extended · Xhigh effort · Cognition-run FrontierCode evaluation; extended set; overall composite score reported as mean@5. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • FrontierCode v1.1 Extended · Max effort · Cognition-run FrontierCode evaluation; extended set; overall composite score reported as mean@5. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • IMO 2026 · proof evaluation · Adaptive thinking at max effort; no tools or agent harness; 256,000-token output limit; four independent solutions per problem. If the output limit was exhausted, attempts were resampled at lower thinking effort; one solution required this. 输出上限 256000 tokens
    未知项 The final 42/42 score aggregates solutions with this stated resampling exception; it is not a uniform single-effort API run.
  • RiemannBench · no tools · Max effort; no tools; private 25-problem research-mathematics benchmark; four attempts per problem; corrected reference answers and grading. max
  • RiemannBench · with tools · Max effort; tools enabled; private 25-problem research-mathematics benchmark; four attempts per problem; corrected reference answers and grading. max
    未知项 The exact tool list is not specified in this section.
  • ArXivMath June 2026 · no tools · Max effort; June 2026 release with 49 problems; four runs per problem; no tools. max
  • ArXivMath June 2026 · with tools · Max effort; June 2026 release with 49 problems; four runs per problem; tools enabled. max
    未知项 The exact tool list is not specified in this section.
  • ProgramBench · single agent · Card-wide default max effort; one agent; 166 retained golden tasks; five sequential episodes, each with a fresh context budget up to 1M tokens; no internet or decompilation tools. max
  • BrowseComp · max effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; Opus 4.7 grader; card-wide default max effort. max
  • OSWorld 2.0 · default evaluation · Card-wide default max effort; default settings at 1080p and up to 500 action steps per task; first-attempt success averaged over five runs; Opus 4.8 grader where required. max
  • HLE effort sweep · no tools · Low effort · No tools; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · no tools · Medium effort · No tools; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · no tools · High effort · No tools; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · no tools · Xhigh effort · No tools; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · no tools · Max effort · No tools; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · with tools · Low effort · Tools enabled; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · with tools · Medium effort · Tools enabled; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · with tools · High effort · Tools enabled; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · with tools · Xhigh effort · Tools enabled; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • HLE effort sweep · with tools · Max effort · Tools enabled; explicit figure-labeled effort scaling; billed API cost reported. This is separate from the headline thinking=auto run. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • BrowseComp · token-budget sweep, unreleased effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; per-task token budgets are recorded in each result protocol; the figure explicitly says the effort configuration was unreleased.
    未知项 The effort level is not disclosed and is not inferred from token budget.
  • BrowseComp · 10M-token budget · Low effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; fixed 10M-token budget; figure-labeled effort; billed API costs. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • BrowseComp · 10M-token budget · Medium effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; fixed 10M-token budget; figure-labeled effort; billed API costs. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • BrowseComp · 10M-token budget · High effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; fixed 10M-token budget; figure-labeled effort; billed API costs. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • BrowseComp · 10M-token budget · Xhigh effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; fixed 10M-token budget; figure-labeled effort; billed API costs. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • BrowseComp · 10M-token budget · Max effort · Single-agent BrowseComp with web search, web fetch, programmatic tool calling, and code execution; context compaction triggered at 200K; fixed 10M-token budget; figure-labeled effort; billed API costs. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSearchQA · Low effort · 900 multi-hop web-research prompts; web search, web fetch, and programmatic tool calling; adaptive thinking; figure states a 980K-token budget; section states a 1M budget and no context compaction. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSearchQA · Medium effort · 900 multi-hop web-research prompts; web search, web fetch, and programmatic tool calling; adaptive thinking; figure states a 980K-token budget; section states a 1M budget and no context compaction. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSearchQA · High effort · 900 multi-hop web-research prompts; web search, web fetch, and programmatic tool calling; adaptive thinking; figure states a 980K-token budget; section states a 1M budget and no context compaction. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSearchQA · Xhigh effort · 900 multi-hop web-research prompts; web search, web fetch, and programmatic tool calling; adaptive thinking; figure states a 980K-token budget; section states a 1M budget and no context compaction. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DeepSearchQA · Max effort · 900 multi-hop web-research prompts; web search, web fetch, and programmatic tool calling; adaptive thinking; figure states a 980K-token budget; section states a 1M budget and no context compaction. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DRACO · Low effort · 100 curated deep-research tasks; web search, web fetch, programmatic tool calling, and code execution; figure states a 980K-token budget; Opus 4.6 grades reports written to a file using the paper formula. Explicit adaptive-thinking effort: low. low
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DRACO · Medium effort · 100 curated deep-research tasks; web search, web fetch, programmatic tool calling, and code execution; figure states a 980K-token budget; Opus 4.6 grades reports written to a file using the paper formula. Explicit adaptive-thinking effort: medium. medium
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DRACO · High effort · 100 curated deep-research tasks; web search, web fetch, programmatic tool calling, and code execution; figure states a 980K-token budget; Opus 4.6 grades reports written to a file using the paper formula. Explicit adaptive-thinking effort: high. high
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DRACO · Xhigh effort · 100 curated deep-research tasks; web search, web fetch, programmatic tool calling, and code execution; figure states a 980K-token budget; Opus 4.6 grades reports written to a file using the paper formula. Explicit adaptive-thinking effort: xhigh. xhigh
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • DRACO · Max effort · 100 curated deep-research tasks; web search, web fetch, programmatic tool calling, and code execution; figure states a 980K-token budget; Opus 4.6 grades reports written to a file using the paper formula. Explicit adaptive-thinking effort: max. max
    未知项 The card does not state any additional sampling parameters beyond the cited evaluation description.
  • Chartography · no tools · Max effort; no tools; 100 chart tasks; internal grading matches the official repository task and expert answer ranges; five-run average. max
  • Chartography · with tools · Max effort; 100 chart tasks; container with image, standard libraries, and image-cropping tool; five-run average. max
  • BenchCAD Vision2Code · no tools · Max effort; 1,000-file random subset of 17,900; voxel IoU; five-run average; no tools. max
  • BenchCAD Vision2Code · with tools · Max effort; 1,000-file random subset of 17,900; container with images, standard libraries, and image-cropping tool; voxel IoU; five-run average. max
  • GDP.pdf · no tools · Max effort; full set of 100 prompts; mean criteria pass rate averaged over five runs; base64-encoded PDFs; PDFs over the API 32MB request limit truncated; Opus 4.7 grader. max
  • GDP.pdf · with tools · Max effort; full set of 100 prompts; container with PDFs, standard libraries, and image-cropping tool; mean criteria pass rate averaged over five runs; Opus 4.7 grader. max
  • OfficeQA / OfficeQA Pro · Messages API · Agentic evaluation with extracted-text documents in a sandbox and code-execution tools; public Messages API; production safety classifiers active with fallback to Opus 4.8 on refusal; card-wide default max effort applies. max
    未知项 The exact run count is not stated in this section.
  • MCP-Atlas · reported production evaluation · Multi-step MCP workflows on production-like servers; the card reports 85.8% pass rate and 89.1% mean claim coverage, and warns that effort settings may since have changed.
    未知项 Effort and run count are not stated; the score may not be precisely reproducible.
  • Legal Agent Benchmark · full set · 1,235 evaluated tasks after 16 data-defect exclusions; adaptive thinking at max effort; public Messages API with production safeguards and fallback to Opus 4.8; internal reimplementation with bash and Python tools; five-run all-pass average; Sonnet 4.6 judge. max
  • Legal Agent Benchmark · Harvey held-out set · External Harvey evaluation on its held-out set; effort, API, toolset, and run count not specified in the system card.
    未知项 Do not combine with Anthropic internal full-set evaluation.
  • GDPval-AA v2 · xhigh · Independent Artificial Analysis evaluation; 220 tasks across 44 occupations and 9 industries; shell and web-browsing agent loop; xhigh effort. xhigh
  • AA-Briefcase · max · Independent Artificial Analysis evaluation; max effort; long-horizon knowledge-work projects; panel grading. max
  • AA-Briefcase · xhigh · Independent Artificial Analysis evaluation; xhigh effort; long-horizon knowledge-work projects; panel grading. xhigh
  • AA-Briefcase · high · Independent Artificial Analysis evaluation; high effort; long-horizon knowledge-work projects; panel grading. high
  • AutomationBench · medium · Zapier private held-out evaluation set; medium effort; result reports $0.89 per task. medium
  • AutomationBench · max · Zapier private held-out evaluation set; max effort; benchmark version and task count not stated. max
  • ARC-AGI-1 · semi-private 2026, max · ARC Prize Foundation semi-private 2026 evaluation set; max effort; verified score. max
    未知项 The card does not state the exact task count or run count.
  • ARC-AGI-2 · semi-private 2026, max · ARC Prize Foundation semi-private 2026 evaluation set; max effort; verified score. max
    未知项 The card does not state the exact task count or run count.
  • HealthBench · max · Max effort; no tools or customized system prompts; Opus 4.8 grader; five-trial average; length-adjustment method cited from the GPT-5.5 system card. max
  • HealthBench Professional · max · Max effort; no tools or customized system prompts; Opus 4.8 grader; five-trial average; length-adjustment method from the HealthBench Professional paper. max
  • GMMLU · max, single trial · Max effort; no tools or customized system prompt; single trial; 42-language overall average. max
  • MILU · max · Max effort; no tools or customized system prompt; five-trial average; 11-language overall average. max
  • INCLUDE · max · Max effort; no tools or customized system prompt; five-trial average; 44 languages. max
  • BioMysteryBench · max · Card-wide default max effort; bash, file editor, and package managers; Human Solvable and Human Difficult subsets; multiple tasks are nonpublic/internal evaluations. max
  • LatchBio Bioinformatics · max · Card-wide default max effort; bash, file editor, and package managers; includes SpatialBench Verified and SingleCellBench. max
  • ProteinGym Hard · max · Card-wide default max effort; bash and file editor; package managers unavailable; rank correlation against lab measurements. max
  • Protein Design · max · Card-wide default max effort; no tool access; constraint satisfaction, folding confidence, and sequence novelty. max
  • Organic chemistry V2 · max · Card-wide default max effort; bash, file editor, and package managers; expert-updated harder problem set. max
  • Protocols · max · Card-wide default max effort; bash, file editor, and web search tools; separate Troubleshooting and Benchling Understanding V2 variants. max

直接前代与后续发布

直接前代 Claude Opus 4.8。 官方关系依据

Claude Opus 5 provides greatly improved performance for the same cost as its predecessor, Opus 4.8.
17 项披露延续 · 29 项新增 · 18 项本次未见 · 0 项待核查
Claude Opus 4.8 与 Claude Opus 5 的披露变化
测试已核查状态Claude Opus 4.8Claude Opus 5
Frontier-Bench · v0.1新增—本次记录
GDPval-AA · v2新增—本次记录
ARC-AGI-3 · 版本/子集未注明新增—本次记录
BrowseComp · 1,266 题;单代理与多代理按协议区分延续

来源位置 attachment → attachment、main

已记录的评测条件有变化

前代记录本次记录
Humanity's Last Exam · 2,500-question benchmark延续

已记录的评测条件有变化

前代记录本次记录
OSWorld · 2.0;Opus 5 发布表,未注明评测子集/协议新增—本次记录
DeepSWE · v1.1新增—本次记录
FrontierCode · v1.1 Main set新增—本次记录
AutomationBench · Zapier 私有 early-access set;版本未注明新增—本次记录
Legal Agent Benchmark · Opus 5 官方发布表所列 held-out set延续

来源位置 attachment → attachment、main

已记录的评测条件有变化

前代记录本次记录
HealthBench Professional · 525 physician-authored conversations延续

来源位置 attachment → attachment、main

已记录的评测条件有变化

前代记录本次记录
BioMysteryBench · 官方发布表列 Hard 与 human-solved 结果;细分版本未注明新增—本次记录
OSWorld · System Card 所称 2.0;2026-09-10 任务与资源版本、108 题、更新 harness新增—本次记录
SWE-bench Verified · Verified subset (500 problems)延续

已记录的评测条件有变化

前代记录本次记录
SWE-bench Pro · Pro延续

来源位置 attachment、main → attachment

已记录的评测条件有变化

前代记录本次记录
SWE-bench Multilingual · 300 problems, 9 programming languages延续

已记录的评测条件有变化

前代记录本次记录
SWE-bench Multimodal · Multimodal延续

已记录的评测条件有变化

前代记录本次记录
FrontierCode · v1.1 Extended set新增—本次记录
International Mathematical Olympiad · 2026 contest; six proof problems新增—本次记录
RiemannBench · 25 private research-mathematics problems; corrected reference answers and grading; four attempts per problem新增—本次记录
ArXivMath · June 2026 release (49 problems)新增—本次记录
ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分新增—本次记录
DeepSearchQA · 900 prompts;单代理与多代理按协议区分延续

已记录的评测条件有变化

前代记录本次记录
DRACO · 100-task Perplexity deep-research benchmark延续

已记录的评测条件有变化

前代记录本次记录
Chartography · 100 tasks新增—本次记录
BenchCAD Vision2Code · 1,000-file random subset of 17,900 files新增—本次记录
GDP.pdf · 100 professional PDFs and prompts新增—本次记录
OfficeQA · Full set延续

已记录的评测条件有变化

前代记录本次记录
OfficeQA · Pro 133-question subset延续

已记录的评测条件有变化

前代记录本次记录
MCP-Atlas · 所选材料未明确子集新增—本次记录
Legal Agent Benchmark · Public full set新增—本次记录
AA-Briefcase · Not stated in selected source新增—本次记录
ARC-AGI-1 · ARC Prize Foundation semi-private 2026 evaluation set新增—本次记录
ARC-AGI-2 · ARC Prize Foundation semi-private 2026 evaluation set新增—本次记录
ARC-AGI-3 · ARC Prize Foundation semi-private 2026 evaluation set新增—本次记录
HealthBench · 48,000+ rubric items across 5,000 multi-turn conversations新增—本次记录
Global MMLU · 42 languages including English · overall average延续

已记录的评测条件有变化

前代记录本次记录
Multi-task Indic Language Understanding Benchmark · 10 Indic languages plus English · overall average延续

已记录的评测条件有变化

前代记录本次记录
INCLUDE · 44 languages延续

已记录的评测条件有变化

前代记录本次记录
SpatialBench Verified · Verified延续

已记录的评测条件有变化

前代记录本次记录
SingleCellBench · LatchBio Bioinformatics single-cell RNA sequencing analysis evaluation延续

已记录的评测条件有变化

前代记录本次记录
ProteinGym Hard · System-card Hard subset; exact task count not stated新增—本次记录
Protein Design Sequence Generation · Protein sequence generation conditioned on design constraints新增—本次记录
Organic Chemistry · V2; system-card evaluation with expert-recommended harder problems新增—本次记录
Protocol Troubleshooting · System-card protocol troubleshooting variant; task count unspecified新增—本次记录
Protocol Understanding · V2; Benchling-created evaluation using real laboratory protocols and applied questions; task count unspecified新增—本次记录
Terminal-Bench · 2.1本次未见前代记录—
OSWorld-Verified · Verified benchmark; evaluation protocol noted per source本次未见前代记录—
GDPval-AA · Artificial Analysis leaderboard本次未见前代记录—
Finance Agent · v2本次未见前代记录—
GPQA Diamond · 198-question Diamond subset本次未见前代记录—
USA Mathematical Olympiad · 2026本次未见前代记录—
ArXivMath · March-April 2026 releases; 71 problems本次未见前代记录—
GraphWalks · 256K-context subset of the 256K-1M evaluation; item count unspecified; BFS F1本次未见前代记录—
GraphWalks · 256K-context subset of the 256K-1M evaluation; item count unspecified; parents本次未见前代记录—
ChartQAPro · ChartQAPro; set size and release version not stated in the System Card本次未见前代记录—
ChartMuseum · 1,162 test-split questions本次未见前代记录—
LAB-Bench FigQA · FutureHouse FigQA; prompting and tools are recorded per result本次未见前代记录—
CharXiv Reasoning · Validation split本次未见前代记录—
ScreenSpot-Pro · 1,581 expert-annotated tasks across 23 professional applications and 3 operating systems本次未见前代记录—
MCP-Atlas · Scale AI 公共排行榜;2026-04 刷新 harness本次未见前代记录—
Vending-Bench · 2本次未见前代记录—
Toolathlon · June 2026 release used in selected source本次未见前代记录—
AutomationBench · Zapier private set; release/version not stated本次未见前代记录—

测试记录 · 46