MirrorCode is not your typical coding benchmark. There are no bug fixes, no function stubs to complete, no isolated snippets. Instead, an AI agent is handed a compiled binary it cannot read, some documentation, and a set of visible test cases -- then told to rebuild the entire program from scratch, in a language it chooses, passing a hidden test suite it will never see during development. Epoch AI just updated the leaderboard with results for Claude Fable 5 and GPT-5.6 Sol, and the gap between them is striking.

What MirrorCode actually measures

MirrorCode is a long-horizon software engineering benchmark that evaluates autonomous coding by requiring AI agents to fully reimplement existing CLI programs. Unlike traditional benchmarks that focus on short code snippets or bug fixes, MirrorCode tasks involve building entire programs from scratch, requiring agents to manage architecture, design, and complex logic over extended periods.

The benchmark was co-developed with METR and supported by a grant from METR. What makes it unusual is the inference budget. Epoch AI provides a large enough inference budget to make a serious attempt at MirrorCode tasks. Many existing software engineering benchmarks limit inference spending to around $1-10, even when the task would take weeks for a human to complete. On MirrorCode, a single run can cost thousands of dollars and run for days.

The setup for each task looks like this:

• The AI has no access to the original source code or the internet. It is given execute-only access to the original compiled binary (a "black box" it can probe), text documentation of the program, and "visible" end-to-end test cases to clarify the exact scope of what needs to be built.
• AI-generated solutions must match the original program's output exactly on end-to-end tests, including held-out tests.
• A solve requires passing 100% of both visible and hidden tests -- no partial credit.