Claude Sonnet 4.6 · BrowseComp · 1,266 题;单代理与多代理按协议区分 · 成绩与配置

Claude Sonnet 4.6 的 BrowseComp · 1,266 题;单代理与多代理按协议区分 记录。本次发布有 6 条成绩记录,含 6 条精确值(可含历史记录),附 6 条来源定位。

成绩与评测条件

Claude Sonnet 4.6 · BrowseComp 成绩与配置
指标与成绩配置与评测条件出处
Accuracy74.72%原文 74.72% · 单位 %
公布 未确认

已更正 · 74.01%。System Card changelog dated 2026-03-06 states that an improved cheating-detection pipeline flagged nine additional unintended solutions; the best-scoring single-agent configuration was re-verified, not rerun.

No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
BrowseComp single-agent best configuration, 1,266 questions.
条件
Historical result recorded in the current System Card changelog before the 2026-03-06 correction. Main article's summary image still shows the rounded 74.7%. No rerun was performed for the single-agent correction.
Accuracy74.01%原文 74.01% · 单位 %
公布

更正记录 · 替代 74.72%。System Card changelog dated 2026-03-06 states that an improved cheating-detection pipeline flagged nine additional unintended solutions; the best-scoring single-agent configuration was re-verified, not rerun.

No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent best configuration, 1,266 questions.
条件
Web search, web fetch, programmatic tool calling, context compaction triggered at 50K tokens, 10M total-token limit, max reasoning effort and no thinking.
在排名中查看此成绩
Accuracy64.69%原文 64.69% · 单位 %
公布 未确认
No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent with 1M sampled-token limit.
条件
Context compaction; the model is told the total-token limit. Web search, web fetch and programmatic tool calling; no thinking.
在排名中查看此成绩
Accuracy69.67%原文 69.67% · 单位 %
公布 未确认
No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent with 3M sampled-token limit.
条件
Context compaction; the model is told the total-token limit. Web search, web fetch and programmatic tool calling; no thinking.
在排名中查看此成绩
Accuracy82.62%原文 82.62% · 单位 %
公布 未确认

已更正 · 82.07%。System Card changelog dated 2026-03-06 states 82.62% -> 82.07%; after removing leaked problems, five of eleven flagged questions remained correct.

Adaptive thinking, max effort
设置
Adaptive thinking enabled; max effort; default temperature and top_p unless an evaluation specifies otherwise.
未知项
Context window is evaluation-dependent; Table 2.1.A states it does not exceed 1M tokens.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
BrowseComp multi-agent orchestrator and subagents.
条件
Historical value in the System Card changelog before the 2026-03-06 update; updated pipeline found 11 unintended solutions.
Accuracy82.07%原文 82.07% · 单位 %
公布

更正记录 · 替代 82.62%。System Card changelog dated 2026-03-06 states 82.62% -> 82.07%; after removing leaked problems, five of eleven flagged questions remained correct.

Adaptive thinking, max effort
设置
Adaptive thinking enabled; max effort; default temperature and top_p unless an evaluation specifies otherwise.
未知项
Context window is evaluation-dependent; Table 2.1.A states it does not exceed 1M tokens.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp multi-agent orchestrator and subagents.
条件
Orchestrator has no direct tools and can only spawn subagents; subagents use web search, web fetch and programmatic tool calling. Orchestrator uses 200K context per subagent, 50K compaction and a 1M total-token limit; max effort. Blacklist updated and affected questions rerun.

出处与披露

测试版本与指标

别名 BrowseComp、Multi-agent BrowseComp

  • Accuracy · % · 越高越好