WebArena-Pro leaderboard

Submit your results

Share your agent’s performance and evaluation traces through a GitHub issue. CI recomputes your success rate from a small scoring archive before maintainer review.

All fields are required unless marked optional. Submit one result per step budget for the complete Test split (300 tasks).

Result

Percentage of the 300 Test tasks with a final reward of 1 at the selected budget. CI verifies this score against your scoring archive.

Evaluation setup

Include your full run configuration and evaluation settings in the trace archive. Identify any retries and which attempts count toward the submitted score.

Agent and resources
Scoring archive Required for CI

Provide a small, public ZIP of scoring JSON files, separate from the full traces. Use the archive creation tool and instructions to package the selected attempt for each of the 300 Test tasks.

CI checks the archive’s SHA-256, task coverage, and step budget, then compares the computed and reported SR to one decimal place. Missing or ungraded scores fail validation. ZIP limit: 25 MiB; no screenshots or large traces are needed for this check.

Use a direct HTTPS download that CI can access without signing in. Pin dataset links to a revision.

Full evaluation traces Required

Link to a downloadable archive or folder with per-task results, complete execution traces, and the run configuration. Include every evaluated task, including failures and interrupted runs.

  • AgentLab: per-task summaries (summary_info.json or summary.json), step logs, and recorded screenshots.
  • Harbor / coding agents: each trial’s result.json, verifier/reward.json for graded trials, agent/ logs (including trajectory.json when produced), and collected WebArena-Pro evidence under artifacts/.

Include recorded screenshots and referenced media. Explain missing or interrupted artifacts; ungraded runs must be distinguishable from scored failures. Make the archive accessible to reviewers.

Use a stable archive or dataset URL. Keep credentials and access tokens out of the submission. A trace viewer can supplement the original artifacts.

Review and submit the prefilled issue on GitHub. CI posts the scoring check result there; maintainers review the evidence before adding your result to the leaderboard.