NVIDIA NeMo Switchyard is a new open-source library that solves one of the most expensive problems in production agentic AI: every step in an agent workflow gets sent to the same frontier model, even when a much cheaper one would do. Long-running agents spend most of their time on tool calls, result validation, and subagent delegation, and sending every one of those steps to a frontier reasoning model adds cost and latency. Switchyard fixes this by routing each step to the model best suited for it, automatically.

NeMo Switchyard is an open-source model routing library for AI agents that routes prompts to the most capable and efficient model for each step of an agent workflow automatically, based on specific needs. It ships alongside Nemotron 3.5 Lightning, a new 30B mixture-of-experts model with only 3B active parameters, purpose-built for the high-volume execution layer of agent pipelines.

The problem every agent builder hits

Some models are better for coding, some for reasoning, some for lightweight tasks, and some are optimized to run locally for greater privacy and efficiency. If you rely on one default model, you might either overspend or lose quality; if you manage routing manually, it becomes integration work that can slow down a deployment. Switchyard is the layer that makes this automatic.

The core insight is that no single model wins on every task. While DeepSeek V4 has the highest overall accuracy on the Terminal-Bench Hard benchmark, it is not the best model for every task group. Kimi K2.6 is better suited to ML and RL task groups, while Qwen3.5 397B A17B is preferable for math and science. A good router exploits these complementary strengths dynamically.

How it works under the hood

Switchyard is built around a provider-agnostic SDK called