NVIDIA has released the Cosmos3-Nano model card, a 16 billion parameter world model that accepts and generates text, images, video, audio, and action trajectories. NVIDIA positions it for Physical AI, its term for systems that perceive and act in environments such as robots, autonomous vehicles, and instrumented spaces.
Action trajectories give the release practical significance for developers. The model can predict future observations from proposed controls, estimate controls from observed video, and generate multimodal training data within one framework. Cosmos3-Nano is the smallest member of the Cosmos 3 family. Its OpenMDW 1.1 license permits commercial and non-commercial use, subject to the license terms.
Two engines split the work
Cosmos3-Nano uses a Mixture-of-Transformers architecture with two complementary generation systems. An autoregressive transformer produces discrete text tokens one at a time. A diffusion transformer generates continuous outputs such as images, video, audio, and actions by repeatedly refining noise into a coherent signal. The components share multimodal representations and conditioning within the same framework.
Action data uses a two-dimensional array shaped (T, D), where T is the number of frames and D is the state dimension for a specific robot or vehicle. Each frame can encode values such as joint positions, gripper state, camera pose, or steering and acceleration commands.
The supplied action schemas cover several embodiments:
- General camera motion and autonomous vehicle control, both 9D
- Egocentric human motion, 57D
- Single and dual Franka Panda arms with RobotiQ grippers, 10D and 20D
- AgiBot humanoid, 29D
- Universal Robots, Google robot, and WidowX 250, each 10D
- UMI robot demonstrations, 9D
The Reasoner path emits structured rationales, subtask plans, 2D or 3D points, and bounding boxes. These outputs can supply a downstream perception or planning system, although generated reasoning traces remain model outputs rather than evidence of reliable internal reasoning.
Benchmarks span reasoning to control
NVIDIA reports comparisons with Gemini, Qwen, and open video-generation models across general reasoning, robotics, smart infrastructure, autonomous driving, and multimodal generation. The published charts place Cosmos3-Nano in the same evaluation set as larger general-purpose systems, although the results remain vendor-reported until independently reproduced.