Gemini 2.5 Pro (GA) · SWE-bench Verified · Verified subset (500 problems) · 成绩与配置

Gemini 2.5 Pro (GA) 的 SWE-bench Verified · Verified subset (500 problems) 记录。本次发布有 2 条成绩记录,含 2 条精确值(可含历史记录),附 4 条来源定位。

成绩与评测条件

Gemini 2.5 Pro (GA) · SWE-bench Verified 成绩与配置
指标与成绩配置与评测条件出处
Pass rate59.6%原文 59.6% · 单位 %
公布 未确认
Gemini 2.5 Pro stable API
设置
AI Studio API model id gemini-2.5-pro; default sampling settings. Technical report states Gemini scores are pass@1 and single-attempt unless specified. Aider Polyglot averages 3 trials; SWE-bench uses the internal agentic harness; Vibe-Eval uses Gemini as judge.
未知项
Thinking budget and any non-default settings for individual evaluations are not stated except where noted in each protocol.
工具
工具条件未记录
推理预算
未记录
框架
未记录
协议
Single agentic trace; internal harness; temperature=1, top-p=0.99, top-k=1024.
条件
AI Studio API model gemini-2.5-pro; default sampling settings unless listed in the benchmark protocol.
在排名中查看此成绩
Pass rate67.2%原文 67.2% · 单位 %
公布 未确认
Gemini 2.5 Pro stable API
设置
AI Studio API model id gemini-2.5-pro; default sampling settings. Technical report states Gemini scores are pass@1 and single-attempt unless specified. Aider Polyglot averages 3 trials; SWE-bench uses the internal agentic harness; Vibe-Eval uses Gemini as judge.
未知项
Thinking budget and any non-default settings for individual evaluations are not stated except where noted in each protocol.
工具
工具条件未记录
推理预算
未记录
框架
未记录
协议
Multiple agentic traces sampled and reranked using Gemini's own judgement; temperature=1, top-p=0.99, top-k=1024.
条件
AI Studio API model gemini-2.5-pro; internal agentic harness.
在排名中查看此成绩

出处与披露

测试版本与指标

别名 SWE-bench Verified

  • Pass rate · % · 越高越好