GLM-5.3 · ExploitGym (2h / 6h) · GLM-5.3模型卡;2h / 6h · 成绩与配置

GLM-5.3 的 ExploitGym (2h / 6h) · GLM-5.3模型卡;2h / 6h 记录。本次发布有 1 条成绩记录,暂无精确值,附 1 条来源定位。

成绩与评测条件

GLM-5.3 · ExploitGym (2h / 6h) 成绩与配置
指标与成绩配置与评测条件出处
pass-at-1105 / 130原文 105 / 130 · 单位 未标注
公布 未确认

原文未提供可确认的精确值。

GLM-5.3 · max
设置
官方模型卡目标列;逐行scaffold、工具、采样及预算条件见protocol。来源把benchmark复现明确绑定max或行内明确reasoning_effort=max。
未知项
未逐项说明的指标单位、题集修订、服务精度和运行次数保持未知。
工具
允许工具
推理预算
max · 输出上限 128000 tokens
框架
Claude Code 2.1.207
协议
The card says reasoning_effort defaults to max and recommends keeping that default for benchmark reproduction. **ExploitGym**: We evaluate GLM-5.3, Kimi-K3 and Qwen3.8 Max in Claude Code 2.1.207 (max reasoning effort, no web tools with `temperature=1.0`, `top_p=1.0`, `max_new_tokens=128000`). The reported results are single-run Pass@1 on 869 tasks under two timeout budgets: 2 hours and 6 hours, which are calculated as the API inference time rescaled by per-model tokens per second rate (per-model TPS sourced from Artificial Analysis; that is, we rescale GLM-5.3's results by 115 TPS, Kimi K3's results by 40 TPS and Qwen3.8 Max's results by 47 TPS), plus the non-API overhead. We also apply a domain whitelist (allowing only essential domains such as pypi.org and deb.debian.org for basic tool installation) to prevent the agent from cheating.
条件
GLM-5.3目标列;The card says reasoning_effort defaults to max and recommends keeping that default for benchmark reproduction. **ExploitGym**: We evaluate GLM-5.3, Kimi-K3 and Qwen3.8 Max in Claude Code 2.1.207 (max reasoning effort, no web tools with `temperature=1.0`, `top_p=1.0`, `max_new_tokens=128000`). The reported results are single-run Pass@1 on 869 tasks under two timeout budgets: 2 hours and 6 hours, which are calculated as the API inference time rescaled by per-model tokens per second rate (per-model TPS sourced from Artificial Analysis; that is, we rescale GLM-5.3's results by 115 TPS, Kimi K3's results by 40 TPS and Qwen3.8 Max's results by 47 TPS), plus the non-API overhead. We also apply a domain whitelist (allowing only essential domains such as pypi.org and deb.debian.org for basic tool installation) to prevent the agent from cheating. 安全、内部子项或跨测试聚合仅保留原始披露,不进入常规展示。
在排名中查看此成绩

出处与披露

测试版本与指标

别名 ExploitGym (2h / 6h)

  • pass-at-1 · 单位未标注 · 越高越好