Documentation
Run the benchmark. Inspect the interaction.
The public ComponentBench repository includes the task pages, Python evaluation harness, task specifications, and human reference traces. Start with a local task, then connect your agent and explore the full protocol.
Quickstart
Node.js 20+ for the task site.
Python 3.11–3.13 for the harness.
1. Start the task site
Clone the released benchmark and run its Next.js app from the repository's site/ directory.
git clone https://github.com/TianchenGuan/ComponentBench
cd ComponentBench/site
npm install
npm run devOpen http://localhost:3002. The first visit to a task can take longer while the development server compiles it.
2. Try a small agent run
Keep the site running. In a second terminal, start from the cloned ComponentBench/ repository root. This example runs two Pixel tasks with an OpenAI model and requires your API key and model access.
pip install -e .
playwright install chromium
export OPENAI_API_KEY="your-api-key"
./scripts/eval_pixel.sh gpt-5.4-mini openai \
--canonical_types button \
--libraries antd \
--max_tasks 2The wrapper configures the provider credentials for the harness; results are written to results/. For other providers, Core evaluations, and production serving, follow the repository quickstart.
Go deeper
These guides live with the released code.
- Evaluation protocol
Run settings, reproducibility, and how to report an evaluation.
- Observation and action interfaces
AX-tree, Set-of-Marks, Pixel, and the separate Browser-Use pipeline.
- Full and Core versions
Choose a benchmark suite and keep task versions explicit.
- Task and result formats
Understand task specifications, metadata, and recorded outputs.
- Human reference recording
Serve the task pages and record cold and warm human attempts locally.
- Failure case studies
Inspect trace-grounded examples and their evidence pointers.
Code and data
Task and run archives, including lightweight traces and full episode packs, are available on Hugging Face. Keep the benchmark version, model version, and observation/action interface with every reported result.
The public repository provides the released benchmark. The Task Lab generation service and the full InterfaceGym platform are maintained separately and are not included in this checkout.