Voxtral 3 Connects Speech Recognition, Synthesis and Tool Calls in One Real-Time Loop

Most teams building voice agents already have a transcription model and a speech synthesis model. The hard part is getting them to work as one loop, where the system hears a caller, decides what to do, calls a tool, and answers before the silence gets awkward. Voxtral 3 is Mistral's attempt to solve that integration problem as well as the component problems. It's a family of speech-to-text and text-to-speech models plus a voice agent SDK, and the pitch is that the four jobs of a voice agent (listen accurately, reason about intent, speak with the right delivery, and act across connected systems) run as a single real-time streaming pipeline.

My read is that the component results matter less here than the architecture decision behind them. Each stage is a separate model, so you can swap the language model, the voice, or the transcription engine without rewriting the agent. That's a different bet from end-to-end speech models, and it has real trade-offs, which I'll come back to.

Transcription that handles 50+ languages and more than one speaker

The speech-to-text side comes in two main variants. Voxtral Small Transcribe 3 is the flagship, with speaker diarization (working out who said what), context biasing, and end-of-turn detection. Voxtral Mini Transcribe 3 is a 4-billion-parameter model aimed at fast, cheap batch work. Mistral says the language list has grown to 50, including 24 official EU languages and a range of lower-resource Asian and Indic languages.

Mistral reports that the Small model outperforms Whisper, Scribe v2 and Gemini 3.5 Transcribe on diarization and multilingual transcription, with English, French and Spanish called out specifically. The benchmark charts in the announcement cover FLEURS in two configurations: one where the language is specified in advance ("forced") and one where it's auto-detected ("inferred"). That split matters in production. Many deployments know the caller's language from the phone number or the user's profile, and many don't, and the gap between those two modes tells you how much a system depends on being told what language to expect.

Context biasing is the feature I'd test first if you run a contact center or a domain-heavy product. You pass in key terms, product names, or speaker names, and the model uses them to improve recall on that vocabulary without fine-tuning. Mistral's keyphrase recall chart on the Earnings-22 benchmark shows results with and without context, with distractor terms ranging from 0 to 100 added to the bias list. That's a sensible test design, because a bias list in production rarely contains only the right words. The uplift from contextual prompting is the number to watch, and whether it holds once your own vocabulary list grows past what the benchmark used.

Diarization is evaluated on a macro average across AMI (both the individual headset and single distant microphone conditions), Switchboard, Switchboard CallHome and AISHELL-4. Mistral notes it used 30-minute versions of the AMI and AISHELL recordings so the API baselines could run under the same conditions. That's a reasonable choice for comparability, but it means the numbers describe 30-minute segments, not the multi-hour sessions some call-center recordings run to.

For streaming, Voxtral Mini Transcribe Realtime 3 is the model built for agentic use. Mistral says it reaches high-quality transcription at sub-500ms latency across more than 35 languages, with diarization and context biasing included. It also natively supports end-of-turn prediction, using both acoustic cues and semantic content to decide when the speaker has finished. Here's the thing about that feature: a pure silence threshold cuts people off mid-sentence whenever they pause to think. Combining sound and meaning should reduce those premature cut-offs, though the announcement gives no false cut-off figures, so you'll need to measure that against your own call audio.

Speech that adjusts emotion, pace and emphasis

Voxtral Mini TTS 3 is the synthesis model, available in batch and realtime modes, and Mistral claims it leads on expressiveness and instruction following across 35+ languages and 50+ dialects. The more interesting design choice is that the model interprets the meaning of the text it's reading to choose delivery. Mistral says it infers emotion and voice style from the prompt's context without extra instructions, which means a sentence of excitement or frustration should come out sounding like it, without markup.

For teams that need control anyway, there's explicit instruction following with variables for emotion, pacing and emphasis. There's also contextual TTS and a vocabulary library for pronunciation. Voice cloning works from 30 minutes or more of sampled audio, and the Pro Voice option is pitched at professional use cases where basic cloning isn't enough. Mistral says Pro Voice can reproduce a speaker's distinctive character, emotional range and speaking style across languages from the same 30 minutes of audio. That's a strong claim, and it's exactly the kind of result to evaluate with listeners who know the original speaker.

The announcement includes side-by-side audio demos comparing Voxtral 1 and Voxtral 3 on customer-support phrasing, an excited congratulations line, an angry complaint, and a flight-booking read-out with numbers and currency. It also includes pronunciation tests where a made-up brand name is given a phoneme hint. These are curated examples, so I'd treat them as illustrations of the direction rather than evidence of consistency across a whole product. Still, the number-and-name cases are where production voice systems usually fail, and it's useful that Mistral chose to demo them.

Mistral also reports a set of internal evaluations with an LLM judge, and this is where I'd slow down. The Voice Code Bench, built by BeSimple AI, uses 300 English prompts focused on numbers and unique names, with two voices per system. A pronunciation evaluation uses 300 sentences, each containing one rare or invented word, split evenly between easy and hard cases. A multilingual evaluation covers 377 prompts across 11 languages, and an expressive English evaluation uses 237 prompts. The charts plot quality against price, and Mistral says Voxtral TTS costs less to run than comparable models from OpenAI TTS-1, Deepgram, ElevenLabs Flash and Gemini Flash. Those comparisons are useful, but the judge model, the prompts and the voices are all chosen or run by Mistral. Treat the rankings as a starting hypothesis and rerun the prompts that look like your traffic.

On language coverage, Voxtral TTS supports over 35 languages and regional dialects. Mistral says its multilingual evaluations account for dialect differences in pronunciation, formal versus informal register, spelling, and locale conventions for times, dates, temperatures, addresses and currencies. That last part is easy to underrate. A voice that reads a price correctly in one locale and stumbles in another is a support ticket waiting to happen. Voice presets are available in 18 languages, including several regional variants such as Emirati Arabic and Brazilian Portuguese.

The agent layer: one pipeline, separate models

The voice agent SDK is built on LiveKit and runs speech in, reasoning and speech out as a streaming pipeline. Voxtral Mini Transcribe Realtime 3 detects when the caller has stopped. A general model, with Mistral Small named in the announcement, handles reasoning and tool calls. Voxtral TTS speaks the reply in the tone the conversation has built up so far. No stage waits for the previous one to finish completely, and Mistral says each turn completes in under one second. The agent also stops talking when the caller interrupts, which is the behaviour that separates a usable phone agent from one that talks over people.

During a call, the agent can verify identity, check a bill or book an appointment through function calling, web search and retrieval over your own systems. Connectivity covers SIP for phone calls and WebRTC for browsers and apps, and the SDK is designed to plug into telephony, IVR and contact-center software that companies already run. The launch demo shows a French-language referral letter being read on the call, a lookup by reference number, an appointment booked from the available slots, and a confirmation text sent back in English. It's a clear picture of the target workflow: a language barrier handled inside the loop, with the system taking actions rather than just answering questions.

The modularity is the part I'd weigh most heavily. If the reasoning model is the weak link, you swap it. If a voice doesn't land with your customers, you change the voice. You can also run the whole stack on Mistral's API in Europe or on your own infrastructure. The cost of that flexibility is integration work. You're responsible for the handoffs between stages, for latency budgets across three models instead of one, and for monitoring each component's failure modes. A single end-to-end speech model would be simpler to operate but harder to inspect and tune when something goes wrong. Teams with strong telephony engineering will likely find the modular design appealing. Teams without that depth should expect to spend time on the plumbing.

Deployment, pricing and what to test first

Voxtral 3 is available now through Mistral's API, with European, US and global endpoints. The Studio playgrounds let you test voice agents, transcription, speech generation and voice cloning before writing any code. For organisations that need to keep audio in their own environment, Mistral offers self-deployment, and the company says it gives control over where audio is processed, how data is handled and whether conversations are retained. Full on-premises or private-cloud deployment goes through a conversation with Mistral's team, so budget time for that process if it's a requirement.

Pricing is per minute for speech-to-text and per character for TTS. The published rates are:

A quick calculation shows where the money goes. A voice agent running on the Mistral Small tier pays for transcription, reasoning and synthesis, so the per-minute agent price is what you'd budget against for a full call, while the component prices matter when you build your own orchestration. Telephony costs from your carrier come on top of all of this, and Mistral's published figures don't include them.

What I'd test first is narrow. Run your own call recordings through the Realtime model with your domain vocabulary as context bias, and measure both recall and the rate of premature turn cut-offs. Then run your most error-prone content through TTS: account numbers, product codes, names, addresses and currency amounts in each language you serve. Mistral's benchmarks are a good map of where to look, but your traffic is the test that counts. The question to answer is whether the modular pipeline's latency and error profile holds on your calls, and if it does, the flexibility to swap models later is the real payoff.