Every voice interaction has a latency budget.
By the time a user hears an application’s response, you have already spent valuable milliseconds on audio capture, speech transcription, LLM execution, context retrieval, response generation, and more. Text-to-speech (TTS) is the final step—and the one users perceive most directly. If speech generation is slow, the entire experience feels sluggish.
The more stages of the pipeline you can run and optimize yourself, the more latency budget you can reclaim.
Voice AI is evolving rapidly. Integrated voice models are simpler—a single API call handles both audio input and audio output—but that simplicity comes at a cost: you cannot fine-tune individual components for your domain, replace models as better ones become available, meet data residency requirements, or determine precisely where latency is coming from. When you need greater control, a cascaded architecture combines specialized ASR, TTS, and LLM components, allowing each layer to be optimized independently and deployed on infrastructure you own.
NVIDIA Magpie Multilingual TTS was built for this purpose. With open weights, production-ready NVIDIA NIM, and support for 12 languages, you can deploy multilingual speech on your own infrastructure, optimize latency for your workloads, and customize the model end to end for specific domains in your own environment.
The latest release expands multilingual coverage by adding Modern Standard Arabic, Korean, and Brazilian Portuguese. It also improves the quality of several existing languages through updated training data and model enhancements.
Whether you are building a customer support agent, medical assistant, enterprise copilot, translation system, or conversational AI application, Magpie provides an open foundation for production-grade voice AI.
Voice AI Is Becoming Multilingual by Default
Today’s voice applications do not serve just one language.
Global customer support, enterprise assistants, medical documentation, retail automation, and translation workflows increasingly require natural conversations across multiple languages while maintaining low latency.
Supporting more languages is only part of the challenge. Developers also need to be able to:
- Deploy where their data resides
- Meet enterprise privacy requirements
- Customize pronunciation and voice characteristics
- Predict latency under production workloads
- Scale on infrastructure they own
Open models change what is possible across all of these dimensions.
One Open Model, Twelve Languages
Magpie TTS Multilingual is an open-weight model with 364 million parameters that supports:
English · Spanish · French · German · Italian · Vietnamese · Mandarin · Hindi · Japanese · Modern Standard Arabic (new) · Korean (new) · Brazilian Portuguese (new)
Each language offers male and female voices through shared multilingual speaker representations.
This release also improves multilingual flexibility by expanding support for code-switching in Hindi and Japanese. The feature uses IPA grapheme-to-phoneme processing and custom pronunciation dictionaries to produce more accurate pronunciations for names, technical terms, and mixed-language content.
Instead of maintaining separate TTS models for different regions, developers can now build multilingual applications on a single open foundation.
Latency Users Can Actually Feel
In conversational AI, text-to-speech is the final stage before users hear a response. As a result, Time to First Audio (TTFA)—the delay between the start of speech generation and the arrival of the first audio segment at the user—is one of the most important latency metrics in the voice pipeline.
Because Magpie TTS can be deployed in your own environment, the latency you measure is the actual server-side latency you can control, excluding the round-trip latency introduced by hosted services.
| GPU | Single-stream TTFA | Single-stream RTFX | 64-stream TTFA | 64-stream RTFX |
|---|---|---|---|---|
| B200 | 32 ms | 12.1× | 239 ms | 319.81× |
| H100 | 47 ms | 14.7× | 275 ms | 290.79× |
| DGX Spark | 53 ms | 9.8× | 962 ms | 75.88× |
| A100 | 79 ms | 12.2× | 395 ms | 197× |
Source: NVIDIA TTS NIM performance documentation (v26.07); figures are averages from three tests conducted in a locally deployed environment.
TTFA = time to first audio; RTFX = throughput expressed as a multiple of real-time speed.
On a B200, Magpie achieves a TTFA of 32 milliseconds, leaving latency budget for the remaining ASR and LLM processing stages and helping keep total end-to-end latency below the 200 milliseconds required for natural conversation. On NVIDIA GPUs, Magpie can achieve single-stream time-to-first-audio latency of 32–79 milliseconds. With 64 concurrent streams, the B200 delivers a TTFA of 239 milliseconds while achieving throughput of 320× real-time speed—meaning that even under concurrent load, audio is generated more than 300 times faster than it is played back.
The table above shows Magpie served through NVIDIA NIM in a locally deployed environment—in other words, an optimized container running on GPUs you own. The open checkpoint on Hugging Face is the same model and provides a path for research and fine-tuning; NIM is the optimized serving stack that delivers the production-grade latency shown above. Both run on hardware under your control.
Because the model runs on your own infrastructure, you can benchmark performance directly, tune it for your deployment environment, and scale it according to your workload. For real-time voice agents, this determines whether a conversation feels responsive or delayed.
Optimized for Real-Time Speech Generation
Low latency does not happen by accident. Magpie introduces two complementary architectural improvements that reduce inference time while preserving speech quality.
Frame stacking. The decoder predicts two audio frames at each decoding step instead of one. This cuts the number of decoder iterations in half, reducing generation time and increasing throughput.
Local transformer. Frame stacking alone introduces dependencies between codebook tokens generated at the same time, which can reduce audio quality. The local transformer models these dependencies and refines the generated audio, compensating for the quality that frame stacking might otherwise sacrifice.
Together, these techniques enable faster generation while still producing natural-sounding speech. The architecture is described in detail in the paper Frame-Stacked Local Transformers for Efficient Multi-Codebook Speech Generation (ICASSP 2026).
Faster Does Not Matter If It Sounds Unnatural
This release does more than add languages—it also improves synthesis quality across several existing languages. Compared with the previous version, Magpie achieves lower character error rates (CER) and higher speaker similarity (SSIM) in multiple languages, with the most significant gains in French and Spanish:
| Language | CER (previous version) | CER (this version) | SSIM (previous version) | SSIM (this version) |
|---|---|---|---|---|
| French | 2.70% | 1.54% | 0.703 | 0.747 |
| Spanish | 1.14% | 0.60% | 0.715 | 0.793 |
| German | 0.66% | 0.80% | 0.626 | 0.742 |
*Source: