LessWrong AI
2026-08-11 20:19 UTC
Score 85.0
USR-0152-20260811-community-fo-ba45551a
Full article
Cross-posted from my Substack . Basically, I created a text-based adventure game benchmark in April, and this morning my agent harness using Claude Opus 5 solved it for the first time. I thought the details might be interesting to this community. First, here are my previous articles on this subject: You’re Standing in a Clearing in a Forest (Apr 10, 2026) Text Adventure Benchmarks Revisited (Jun 14, 2026) Testing Fable 5 on Text Adventure Games (Jul 5, 2026) And a reminder of the domain. This is a small custom text-based adventure I created from scratch as a personal benchmark to run new models of LLMs against. It’s 10 rooms total and the goal is to collect 3 keys (brass, silver, and gold) and use them correctly to unlock the final door in Room 3 to exit the dungeon. The first and most challenging central puzzle is a rotating room (r5) operated by a crank mechanism in r4. The player must first find the handle to the crank in r2, carry it to r4, insert it, and turn it to align openings between r5 and its adjacent rooms. The most difficult aspect of this puzzle seemed to be non-local causal reasoning combined with allocentric coordinates. The crank is two rooms away from the rotating room that it actually turns. When the player turns the crank a grinding sound nearby can be heard through the walls. There is an informational diagram on the wall in the same room as the wall, and it updates with each turn. Earlier models struggled to understand that the diagram was information, a…
