New NVIDIA Cosmos 3 Open Models for Interactive World Simulation

Why it matters

NVIDIA has released four open-weight checkpoints extending its Cosmos 3 model family for interactive world simulation and multi-sensor driving scenes. All four are post-trained from Cosmos 3 Nano and share a common development path: Cosmos3-Nano-Transfer-Auto generates synchronized multi-view driving data, including fisheye cameras and joint RGB–LiDAR output, guided by World Scenario Map controls and text prompts. It can be adapted to a developer's own sensor rig and then distilled into Cosmos3-Nano-Sim-Auto, a causally trained streaming variant that produces successive chunks of synchronized camera and LiDAR observations for long interactive rollouts. Cosmos3-Nano-Sim-Transfer converts a physics simulator's spatial controls—depth, segmentation, edges, and blur—plus a text prompt into photorealistic color video, keeping physics with the simulator. Cosmos3-Nano-Sim-Bimanual predicts video from a two-armed robot's actions, supporting YAM and AgiBot embodiments under policy or joystick control. The release is framed as an open foundation for research, with the hard work of validating these models for policy learning still ahead.