Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS

Why it matters

NVIDIA has released NVIDIA Magpie Multilingual TTS, an open-weights text-to-speech model designed for voice agents that need low latency and on-premise control. The model supports twelve languages, adding Modern Standard Arabic, Korean, and Brazilian Portuguese to its existing coverage, with male and female voices per language through a shared speaker representation. It also expands code-switching support for Hindi and Japanese using IPA grapheme-to-phoneme processing and custom pronunciation dictionaries. The article frames the release around deployment control: rather than a managed integrated speech API, developers can run the model via NVIDIA NIM on their own infrastructure, benchmark latency themselves, fine-tune for their domain, and enforce data residency. Architecturally, the model pairs frame stacking, which halves decoder iterations by predicting two audio frames per step, with a local transformer that preserves speech quality. NVIDIA reports first audio in roughly the time of a quick blink on B200 GPUs, keeping total conversational latency within a natural-feeling window. The work is documented in an ICASSP paper.