Humanity’s Last Exam; reasoning only, no tools; auto thinking and max effort; 1M total-token cap; no context compaction. Some items exceed the Public API one-hour sampling limit.
未知项
具体题集与试验数未列于主表。
工具
禁止工具
推理预算
max
框架
未记录
协议
Humanity’s Last Exam; reasoning only, no tools; auto thinking, max effort; 1M total-token cap; no context compaction.
条件
The System Card says some no-tools problems exceed the Public API one-hour sampling limit; retain the reported 49.8% and source evidence with this reproducibility limitation.
Humanity’s Last Exam; web search, web fetch, programmatic tool calling and code execution; auto thinking and max effort; 1M total-token cap; no context compaction; Opus 4.6 transcript review/regrade for confirmed answer retrieval.
未知项
具体题集与试验数未列于主表。
工具
允许工具
推理预算
max
框架
未记录
协议
Humanity’s Last Exam with web search, web fetch, programmatic tool calling and code execution; auto thinking, max effort; 1M total-token cap; no context compaction.
条件
Opus 4.6 transcript review and regrade were used to confirm HLE answer retrieval.
Claude Opus 4.8 · Humanity's Last Exam · 2,500-question benchmark · Scores and configurations
Claude Opus 4.8 record for Humanity's Last Exam · 2,500-question benchmark. 2 score records for this release, including 2 exact values (may include historical records). Includes 3 source locations.
Claude Opus 4.8 · Humanity's Last Exam scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Accuracy49.8%Reported 49.8% · Unit %
Published Unconfirmed
HLE 无工具
Settings
Humanity’s Last Exam; reasoning only, no tools; auto thinking and max effort; 1M total-token cap; no context compaction. Some items exceed the Public API one-hour sampling limit.
Unknowns
具体题集与试验数未列于主表。
Tools
Tools forbidden
Reasoning budget
max
Framework
not recorded
Protocol
Humanity’s Last Exam; reasoning only, no tools; auto thinking, max effort; 1M total-token cap; no context compaction.
Conditions
The System Card says some no-tools problems exceed the Public API one-hour sampling limit; retain the reported 49.8% and source evidence with this reproducibility limitation.
Humanity’s Last Exam; web search, web fetch, programmatic tool calling and code execution; auto thinking and max effort; 1M total-token cap; no context compaction; Opus 4.6 transcript review/regrade for confirmed answer retrieval.
Unknowns
具体题集与试验数未列于主表。
Tools
Tools allowed
Reasoning budget
max
Framework
not recorded
Protocol
Humanity’s Last Exam with web search, web fetch, programmatic tool calling and code execution; auto thinking, max effort; 1M total-token cap; no context compaction.
Conditions
Opus 4.6 transcript review and regrade were used to confirm HLE answer retrieval.