---
title: "ClawBench: AI Browser Agent Benchmark on Live Websites"
url: https://claw-bench.com/
updated: 2026-05-20
type: WebSite
---

# ClawBench

ClawBench is an open benchmark that measures whether AI browser agents can complete everyday online tasks on live websites. The V2 corpus has 130 tasks across 63 platforms in 30 task categories, and the V1 corpus has 153 tasks. Each run is scored in two stages: the agent's final HTTP request is intercepted and matched to the task schema, then an LLM judge checks the payload against the instruction. On V2 (hermes harness), claude-opus-4-7 leads with 71 of 130 runs intercepted (54.6%) and a 44.6% reward rate; on V1, claude-opus-4-6 leads with 94 of 153 (61.4%). Leaderboard data as of 2026-05-20.

Paper: [ClawBench: Can AI Agents Complete Everyday Online Tasks?](https://arxiv.org/abs/2604.08523) (arXiv:2604.08523). Read it with `hf papers read 2604.08523`.

## V2 leaderboard

| Rank | Model | Harness | Tasks | Intercepted | Intercepted rate | Reward rate |
|---|---|---|---|---|---|---|
| 1 | [claude-opus-4-7](https://claw-bench.com/models/claude-opus-4-7/index.md) | hermes | 130 | 71 | 54.6% | 44.6% |
| 2 | [gpt-5.5](https://claw-bench.com/models/gpt-5.5/index.md) | hermes | 130 | 59 | 45.4% | 35.4% |
| 3 | [glm-5.1](https://claw-bench.com/models/glm-5.1/index.md) | hermes | 130 | 63 | 48.5% | 34.6% |
| 4 | [deepseek-v4-pro](https://claw-bench.com/models/deepseek-v4-pro/index.md) | hermes | 130 | 57 | 43.9% | 33.9% |
| 5 | [deepseek-v4-flash:free](https://claw-bench.com/models/deepseek-v4-flash-free/index.md) | hermes | 130 | 4 | 3.1% | 2.3% |
| 6 | [z-ai/glm-4.5-air:free](https://claw-bench.com/models/z-ai-glm-4.5-air-free/index.md) | hermes | 130 | 6 | 4.6% | 2.3% |
| 7 | [minimax-m2.5:free](https://claw-bench.com/models/minimax-m2.5-free/index.md) | hermes | 130 | 3 | 2.3% | 1.5% |
| 8 | [openrouter-owl-alpha](https://claw-bench.com/models/openrouter-owl-alpha/index.md) | hermes | 130 | 19 | 14.6% | 0.0% |
| 9 | [glm-5.1](https://claw-bench.com/models/glm-5.1/index.md) | openclaw | 130 | 0 | 0.0% | 0.0% |

## V1 leaderboard

| Rank | Model | Harness | Tasks | Intercepted | Intercepted rate | Reward rate |
|---|---|---|---|---|---|---|
| 1 | [claude-opus-4-6](https://claw-bench.com/models/claude-opus-4-6/index.md) | hermes | 153 | 94 | 61.4% | 61.4% |
| 2 | [claude-sonnet-4-6](https://claw-bench.com/models/claude-sonnet-4-6/index.md) | hermes | 153 | 87 | 56.9% | 56.9% |
| 3 | [claude-haiku-4-5-20251001](https://claw-bench.com/models/claude-haiku-4-5-20251001/index.md) | hermes | 153 | 46 | 30.1% | 30.1% |
| 4 | [gpt-5.4-2026-03-05](https://claw-bench.com/models/gpt-5.4-2026-03-05/index.md) | hermes | 153 | 39 | 25.5% | 25.5% |
| 5 | [gpt-5.4-mini-2026-03-17](https://claw-bench.com/models/gpt-5.4-mini-2026-03-17/index.md) | hermes | 153 | 38 | 24.8% | 24.8% |
| 6 | [kimi-k2.5](https://claw-bench.com/models/kimi-k2.5/index.md) | hermes | 153 | 27 | 17.6% | 17.6% |

## FAQ

### How many tasks and platforms does ClawBench cover?

The V2 corpus has 130 tasks on 63 live platforms in 30 categories; the V1 corpus has 153 tasks.

### Which model leads the ClawBench leaderboard?

On V2 (hermes harness), claude-opus-4-7 leads with 71 of 130 runs intercepted (54.6%) and a 44.6% reward rate. Data as of 2026-05-20.

### How is the leaderboard scored and ranked?

Two stages. Stage 1 (Intercepted, deterministic) : the run counts only if the agent's final HTTP request matches the task's URL/method schema. Stage 2 (Reward) : a judge model ( deepseek/deepseek-v4-pro ) reads the intercepted payload against the instruction — lenient rubric ("no contradiction → match") for the headline Reward, strict rubric ("ambiguous → mismatch") for Reward (strict), visible via the "All columns" toggle. Ranked by Reward (lenient). Details: eval/scoring.md ↗

### How does ClawBench relate to claw-eval, WildClawBench, and ClawMark?

ClawBench (this site, arXiv:2604.08523 ) focuses on live-website agent tasks with HTTP-request interception + LLM-judge scoring. One codebase runs every benchmark : the ClawBench harness supports them as adapters (different task corpora, same evaluator, same trace bundle), alongside scope-peers like WebVoyager and OSWorld . claw-eval/claw-eval — 300 human-verified tasks, 9 categories, Pass^3 scoring. Adapter: #170 InternLM/WildClawBench — 60 in-the-wild tasks in a live OpenClaw environment. Adapter: #171 evolvent-ai/ClawMark — 100 multi-day coworker-agent tasks, deterministic Python checkers. Adapter: #169

### How do I submit my model?

Run the CLI ( pip install clawbench-eval && clawbench run --corpus v2 --model your-model ) and open a PR with your run artifacts — github.com/TIGER-AI-Lab/ClawBench/pulls ↗ . Or start from the Contribute page.

## Cite

```bibtex
@article{zhang2026clawbench,
  title={ClawBench: Can AI Agents Complete Everyday Online Tasks?},
  author={Yuxuan Zhang and Yubo Wang and Yipeng Zhu and Penghui Du and Junwen Miao and Xuan Lu and Wendong Xu and Yunzhuo Hao and Songcheng Cai and Xiaochen Wang and Huaisong Zhang and Xian Wu and Yi Lu and Minyi Lei and Kai Zou and Huifeng Yin and Ping Nie and Liang Chen and Dongfu Jiang and Wenhu Chen and Kelsey R. Allen},
  journal={arXiv preprint arXiv:2604.08523},
  year={2026}
}
```
