工具条件未记录 / max / mini-SWE-agent harness · Single-agent ProgramBench; 166 retained golden tasks; mini-swe-agent harness; no six-hour time limit; no internet or decompilation.
ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Hidden test pass rate · 工具条件未记录 / max / mini-SWE-agent harness · Single-agent ProgramBench; 166 retained golden tasks; mini-swe-agent harness; no six-hour time limit; no internet or decompilation.
Regular display scope. Rankings are grouped by metric and evaluation protocol. Recorded tool access, budgets, and frameworks are shown separately. Scores remain available for reference when conditions are unspecified.
Hidden test pass rate %
Tool access not recorded / max / mini-swe-agent · ProgramBench; hidden-test pass rate on 166 retained tasks using mini-swe-agent harness.
ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Hidden test pass rate · Tool access not recorded / max / mini-swe-agent · ProgramBench; hidden-test pass rate on 166 retained tasks using mini-swe-agent harness.
Tool access not recorded / max / Framework not recorded · ProgramBench, 166 tasks after excluding 34 flaky reference binaries; hidden-test pass rate over five episodes.
ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Hidden test pass rate · Tool access not recorded / max / Framework not recorded · ProgramBench, 166 tasks after excluding 34 flaky reference binaries; hidden-test pass rate over five episodes.
Tool access not recorded / max / mini-SWE-agent harness · Single-agent ProgramBench; 166 retained golden tasks; mini-swe-agent harness; no six-hour time limit; no internet or decompilation.
ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Hidden test pass rate · Tool access not recorded / max / mini-SWE-agent harness · Single-agent ProgramBench; 166 retained golden tasks; mini-swe-agent harness; no six-hour time limit; no internet or decompilation.