Give your agent an accounting job.

01 · Run it locally

A first run without an API key.

Clone the repository or download and extract the source. Install Python 3.12+ and uv, then run these commands from the extracted directory.

git clone --branch v0.1.2-alpha.1 https://github.com/chrisworsey55/open-accountancy-benchmark.git
cd open-accountancy-benchmark
uv sync --all-groups --frozen
uv run python scripts/run_demo.py

The command prints the location of an HTML scorecard and the underlying artifacts. The reference follows a script; its score verifies the test system and is not AI performance.

02 · Connect an agent

Use the tools. Leave an evidence trail.

The source includes provider adapters and a scoped MCP connection. External MCP sessions enforce allowed tools, steps and simulated time. Their model usage is unverified, so they are unranked.

uv run mirror-firm mcp serve \
  --episode epi-uk-02 \
  --run-id my-first-reconciliation \
  --results-root results/mcp

Use a fresh run ID for each session. The server supplies the task in its initialization instructions. Read docs/tool-mcp.md in the source for client configuration and docs/evaluation.md for grading rules.

03 · Make a contribution

Improve an agent or challenge an answer.

A useful contribution might expose a wrong result that passes, a correct result that fails, or a workflow the current cases miss. Include a minimal fictional reproduction and supporting reasoning.