Benchmarking Jev against no-CoT LLMs

TL;DR. We ran Jev 1.13 — TypeSafe's non-autoregressive model, which answers questions with probabilities and cannot emit text, on the ThinkFast no-CoT suite and Neel Nanda’s NCRI . Jev’s capabilities are very jagged. As a transcript monitor its AUROC on sabotage and sandbagging detection is level with GPT-5.5 and Opus 4.7 without CoT , at about 1/200th of Opus 4.7’s cost. On multiple-choice questions it ranges from ahead of GPT-5.5 (GPQA-Diamond) to well behind it (Sally-Anne). It does badly on tasks where the answer involves multi-step reasoning: on NCRI, it ranks 209 of 279, at about GPT 3.5 level. This matters because cost and speed make Jev an appealing option for two jobs : routing and simple decision-making in agentic systems, and always-on monitoring of agents. In both, its decisions leave no reasoning trace to inspect. It could also be a cheap test bed for developing frontier monitors, if improvements on Jev carry over. We thank Rauno Arike, Jonathan Bostock, Andrew Draganov, Francis Rhys Ward and Nicole Xing for helpful comments and discussions. Background: Jev TypeSafe calls Jev a “ System One ” model, pitching it for decisions like classification, routing, scoring, and verification. Unlike standard LLMs, Jev doesn’t take a prompt and return a token-by-token output. Instead, you send it a state , which is text describing the situation (e.g. a problem statement/ agent’s transcript), together with one or more typed questions about that state. Jev answers all of them…