Card-wide default max effort; one agent; 166 retained golden tasks; five sequential episodes, each with a fresh context budget up to 1M tokens; no internet or decompilation tools.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
ProgramBench; 166 golden tasks; hidden-test pass rate after episode 1.
条件
Single-agent; episode 1; each episode has a fresh context budget up to 1M tokens; max effort by card-wide default.
Card-wide default max effort; one agent; 166 retained golden tasks; five sequential episodes, each with a fresh context budget up to 1M tokens; no internet or decompilation tools.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
ProgramBench; 166 golden tasks; hidden-test pass rate after episode 5.
条件
Single-agent; five sequential episodes continue from the previous codebase; fresh context budget up to 1M tokens per episode; max effort by card-wide default.
Claude Opus 5 · ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Scores and configurations
Claude Opus 5 record for ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分. 2 score records for this release, including 2 exact values (may include historical records). Includes 1 source location.
Claude Opus 5 · ProgramBench scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Hidden test pass rate83%Reported 83% · Unit %
Published Unconfirmed
ProgramBench · single agent
Settings
Card-wide default max effort; one agent; 166 retained golden tasks; five sequential episodes, each with a fresh context budget up to 1M tokens; no internet or decompilation tools.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
ProgramBench; 166 golden tasks; hidden-test pass rate after episode 1.
Conditions
Single-agent; episode 1; each episode has a fresh context budget up to 1M tokens; max effort by card-wide default.
Card-wide default max effort; one agent; 166 retained golden tasks; five sequential episodes, each with a fresh context budget up to 1M tokens; no internet or decompilation tools.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
ProgramBench; 166 golden tasks; hidden-test pass rate after episode 5.
Conditions
Single-agent; five sequential episodes continue from the previous codebase; fresh context budget up to 1M tokens per episode; max effort by card-wide default.