Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Toolathlon-Verified June 2026; 108 tasks; Pass@1 averaged over three trials per task; adaptive thinking/max effort.
条件
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Toolathlon-Verified June 2026; 108 tasks; at least one of three trials correct; adaptive thinking/max effort.
条件
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Toolathlon-Verified June 2026; 108 tasks; all three trials correct; adaptive thinking/max effort.
条件
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Average assistant turns26.9 turns原文 26.9 turns · 单位 turns
公布 未确认
Toolathlon-Verified max effort with safeguards
设置
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
未知项
未另记
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Toolathlon-Verified June 2026; average assistant turns per trajectory across 108 tasks and three trials.
条件
Trajectory length is a separate reported metric, not a success rate. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Claude Opus 5.5 · Toolathlon-Verified · June 2026 finalized release · Scores and configurations
Claude Opus 5.5 record for Toolathlon-Verified · June 2026 finalized release. 4 score records for this release, including 4 exact values (may include historical records). Includes 1 source location.
Claude Opus 5.5 · Toolathlon-Verified scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Pass@177.8%Reported 77.8% Pass@1 · Unit %
Published Unconfirmed
Toolathlon-Verified max effort with safeguards
Settings
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
Toolathlon-Verified June 2026; 108 tasks; Pass@1 averaged over three trials per task; adaptive thinking/max effort.
Conditions
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
Toolathlon-Verified June 2026; 108 tasks; at least one of three trials correct; adaptive thinking/max effort.
Conditions
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
Toolathlon-Verified June 2026; 108 tasks; all three trials correct; adaptive thinking/max effort.
Conditions
Production safety classifier and sandbox-escape monitor enabled; all stopped trials counted as failures. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.
Average assistant turns26.9 turnsReported 26.9 turns · Unit turns
Published Unconfirmed
Toolathlon-Verified max effort with safeguards
Settings
Adaptive thinking/max effort; production safety classifiers and sandbox-escape monitor enabled; three trials per task; all stopped trials counted as failures.
Unknowns
none recorded
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
Toolathlon-Verified June 2026; average assistant turns per trajectory across 108 tasks and three trials.
Conditions
Trajectory length is a separate reported metric, not a success rate. Source-wide conditions: Toolathlon-Verified June 2026; 108 tasks; three trials per task; adaptive thinking/max effort; safety classifiers enabled; one classifier stop and six sandbox-escape-monitor stops counted as failures.