Claude Sonnet · BrowseComp · 1,266 题;单代理与多代理按协议区分 · 历代记录

按发布时间查看 Claude Sonnet 的 2 次发布在 BrowseComp · 1,266 题;单代理与多代理按协议区分 的成绩、配置与披露。

仅为直接前代间的唯一、有效精确值计算差值,要求单位、指标与明确记录的配置和条件一致。

Claude Sonnet 4.6

· 本次发布有 6 条成绩记录,含 6 条精确值(可含历史记录)

Claude Sonnet 4.6 · BrowseComp 成绩与配置
指标与成绩配置与评测条件出处
Accuracy74.72%原文 74.72% · 单位 %
公布 未确认

已更正 · 74.01%。System Card changelog dated 2026-03-06 states that an improved cheating-detection pipeline flagged nine additional unintended solutions; the best-scoring single-agent configuration was re-verified, not rerun.

No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
BrowseComp single-agent best configuration, 1,266 questions.
条件
Historical result recorded in the current System Card changelog before the 2026-03-06 correction. Main article's summary image still shows the rounded 74.7%. No rerun was performed for the single-agent correction.
Accuracy74.01%原文 74.01% · 单位 %
公布

更正记录 · 替代 74.72%。System Card changelog dated 2026-03-06 states that an improved cheating-detection pipeline flagged nine additional unintended solutions; the best-scoring single-agent configuration was re-verified, not rerun.

No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent best configuration, 1,266 questions.
条件
Web search, web fetch, programmatic tool calling, context compaction triggered at 50K tokens, 10M total-token limit, max reasoning effort and no thinking.
在排名中查看此成绩
Accuracy64.69%原文 64.69% · 单位 %
公布 未确认
No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent with 1M sampled-token limit.
条件
Context compaction; the model is told the total-token limit. Web search, web fetch and programmatic tool calling; no thinking.
在排名中查看此成绩
Accuracy69.67%原文 69.67% · 单位 %
公布 未确认
No thinking, max effort
设置
Thinking disabled; max effort.
未知项
Sampling settings are not specified in every evaluation.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp single-agent with 3M sampled-token limit.
条件
Context compaction; the model is told the total-token limit. Web search, web fetch and programmatic tool calling; no thinking.
在排名中查看此成绩
Accuracy82.62%原文 82.62% · 单位 %
公布 未确认

已更正 · 82.07%。System Card changelog dated 2026-03-06 states 82.62% -> 82.07%; after removing leaked problems, five of eleven flagged questions remained correct.

Adaptive thinking, max effort
设置
Adaptive thinking enabled; max effort; default temperature and top_p unless an evaluation specifies otherwise.
未知项
Context window is evaluation-dependent; Table 2.1.A states it does not exceed 1M tokens.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
BrowseComp multi-agent orchestrator and subagents.
条件
Historical value in the System Card changelog before the 2026-03-06 update; updated pipeline found 11 unintended solutions.
Accuracy82.07%原文 82.07% · 单位 %
公布

更正记录 · 替代 82.62%。System Card changelog dated 2026-03-06 states 82.62% -> 82.07%; after removing leaked problems, five of eleven flagged questions remained correct.

Adaptive thinking, max effort
设置
Adaptive thinking enabled; max effort; default temperature and top_p unless an evaluation specifies otherwise.
未知项
Context window is evaluation-dependent; Table 2.1.A states it does not exceed 1M tokens.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp multi-agent orchestrator and subagents.
条件
Orchestrator has no direct tools and can only spawn subagents; subagents use web search, web fetch and programmatic tool calling. Orchestrator uses 200K context per subagent, 50K compaction and a 1M total-token limit; max effort. Blacklist updated and affected questions rerun.

Claude Sonnet 5

· 本次发布有 3 条成绩记录,含 3 条精确值(可含历史记录)

与直接前代的记录缺少满足差值计算的同条件依据。

Claude Sonnet 5 · BrowseComp 成绩与配置
指标与成绩配置与评测条件出处
Accuracy84.7%原文 84.7% · 单位 %
公布 未确认
Standard configuration: adaptive thinking, max effort
设置
Adaptive thinking at max effort; default temperature and top_p; five-trial averages unless otherwise stated.
未知项
Context-window size is evaluation-dependent; standard configuration is 1M tokens.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp with web search, web fetch, programmatic tool calling and code execution.
条件
Adaptive thinking, max effort, 10M token limit, 200K context compaction, evaluation blocklist. Original page chart was replaced after the source identified a simpler methodology; prior numeric value was not given and is not reconstructed.
在排名中查看此成绩
Accuracy79.3%原文 79.3% · 单位 %
公布 未确认
BrowseComp max effort, 1M-token limit
设置
Adaptive thinking at max effort with web search, web fetch, programmatic tool calling, and code execution; 1M-token limit.
未知项
Whether context compaction was used at this token limit is not stated.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp Figure 8.9.2.A, test-time compute scaling at max effort with web search, web fetch, programmatic tool calling, and code execution.
条件
1M-token limit as labeled. The figure gives no exact per-point cost label.
Accuracy82.9%原文 82.9% · 单位 %
公布 未确认
BrowseComp max effort, 3M-token limit
设置
Adaptive thinking at max effort with web search, web fetch, programmatic tool calling, and code execution; 3M-token limit; context compaction is reported as triggering at 200K tokens when extending beyond the 1M context window.
未知项
Sampling parameters are not stated.
工具
允许工具
推理预算
max
框架
未记录
协议
BrowseComp Figure 8.9.2.A, test-time compute scaling at max effort with web search, web fetch, programmatic tool calling, and code execution.
条件
3M-token limit as labeled; §8.9.2 reports context compaction triggered at 200K tokens to extend beyond the 1M context window. The figure gives no exact per-point cost label.