Legal Agent Benchmark held-out problem set; standard all-criteria-pass task success.
条件
1,235 problems tested; 16/1,251 removed for data defects before testing; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge.
Legal Agent Benchmark held-out problem set; mean rubric-criterion pass rate.
条件
1,235 problems tested; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge. Metric is a proposed extension to the held-out catalog entry.
System Card max effort (per-result benchmark harnesses)
设置
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
未知项
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
Distinct from the public full LAB set; do not label as Harvey's public leaderboard score. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.
Mean criterion-pass rate91.2%原文 91.2% mean criterion-pass rate · 单位 %
公布 未确认
System Card max effort (per-result benchmark harnesses)
设置
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
未知项
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
Criterion-pass rate is not the all-pass rate; keep the two metrics separate. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.
Claude Opus · Legal Agent Benchmark · Opus 5 官方发布表所列 held-out set · Release history
View scores, configurations, and disclosures for Legal Agent Benchmark · Opus 5 官方发布表所列 held-out set across 3 Claude Opus releases, ordered by release date.
Differences are calculated only for unique, active, exact values between direct predecessors when the unit, metric, and explicitly recorded configuration and conditions match.
Legal Agent Benchmark held-out problem set; standard all-criteria-pass task success.
Conditions
1,235 problems tested; 16/1,251 removed for data defects before testing; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge.
Legal Agent Benchmark held-out problem set; mean rubric-criterion pass rate.
Conditions
1,235 problems tested; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge. Metric is a proposed extension to the held-out catalog entry.
· 2 score records for this release, including 2 exact values (may include historical records)
Records for the direct predecessor do not provide matching conditions for a valid difference.
Claude Opus 5.5 · Legal Agent Benchmark scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
All-pass rate8.3%Reported 8.3% all-pass rate · Unit %
Published Unconfirmed
System Card max effort (per-result benchmark harnesses)
Settings
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
Unknowns
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
Distinct from the public full LAB set; do not label as Harvey's public leaderboard score. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.
Mean criterion-pass rate91.2%Reported 91.2% mean criterion-pass rate · Unit %
Published Unconfirmed
System Card max effort (per-result benchmark harnesses)
Settings
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
Unknowns
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
Criterion-pass rate is not the all-pass rate; keep the two metrics separate. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.