Documentation

Run the benchmark. Inspect the interaction.

The public ComponentBench repository includes the task pages, Python evaluation harness, task specifications, and human reference traces. Start with a local task, then connect your agent and explore the full protocol.

Quickstart

Node.js 20+ for the task site.
Python 3.11–3.13 for the harness.

1. Start the task site

Clone the released benchmark and run its Next.js app from the repository's site/ directory.

git clone https://github.com/TianchenGuan/ComponentBench
cd ComponentBench/site
npm install
npm run dev

Open http://localhost:3002. The first visit to a task can take longer while the development server compiles it.

2. Try a small agent run

Keep the site running. In a second terminal, start from the cloned ComponentBench/ repository root. This example runs two Pixel tasks with an OpenAI model and requires your API key and model access.

pip install -e .
playwright install chromium
export OPENAI_API_KEY="your-api-key"

./scripts/eval_pixel.sh gpt-5.4-mini openai \
  --canonical_types button \
  --libraries antd \
  --max_tasks 2

The wrapper configures the provider credentials for the harness; results are written to results/. For other providers, Core evaluations, and production serving, follow the repository quickstart.

Go deeper

These guides live with the released code.

Code and data

Task and run archives, including lightweight traces and full episode packs, are available on Hugging Face. Keep the benchmark version, model version, and observation/action interface with every reported result.

The public repository provides the released benchmark. The Task Lab generation service and the full InterfaceGym platform are maintained separately and are not included in this checkout.