With Realtime API instead of separate speech-to-text, LLM, and text-to-speech components, the model receives call audio directly and responds with audio. That simplicity comes with a question worth asking early: where does that audio go?
Unlike text-based LLMs where the sensitive data is first converted to a text transcript, Realtime API sends raw audio directly to the model. That input audio contains the caller’s voice characteristics, background sounds that may add context, and the full content of the call. The provider you choose and where they host their endpoints determines where all of that is processed.
We covered text-based LLMs and data residency in the previous article.
Controlling where audio goes
VoxEngine gives you control over this through the same mechanism as text-based LLMs: the baseUrl parameter in the Realtime API client. Point it at any OpenAI Realtime-compatible endpoint and that's where call audio goes — a cloud provider with regional deployments, or your own infrastructure.
Azure OpenAI offers Realtime model deployments across multiple regions including EU-based datacenters. If your clients require data to stay within EU infrastructure, that gives you a documented answer to where audio is processed. One detail worth knowing before going to production: deployment type matters. Global deployments can process audio in any Azure region where the model is available, even if your resource is located in the EU.
- Confirm that Azure currently supports your Realtime model in the target EU region with the right deployment type.
- Select the Azure resource and endpoint for that region and set it as baseUrl.
- Choose a deployment type with the right processing boundary (Data Zone EU or Regional).
Full details and a working VoxEngine scenario.
What else to verify before going to production
Realtime API providers vary significantly in how they handle audio data. Before routing live calls through any endpoint, check:
- Audio logging and retention policy. Some providers store audio for model improvement by default. Check if you can opt out.
- Compliance documentation. DPA availability, sub-processor lists, breach notification terms matter as much for audio as for text.
The tradeoff
Realtime API handles the full voice pipeline in one model. That reduces moving parts but it also means less flexibility: you get the model's built-in voice and language support rather than independently chosen components.
If your product requirements or compliance situation call for specific STT or TTS providers or controls on what information is sent to the model, a full-cascade pipeline gives you more control over each layer. Both approaches support data residency through baseUrl. The choice depends on what else you need from your stack.
This is the second post in a series on building compliant voice AI on Voximplant. Next is a complete guide to running a fully sovereign voice AI stack end to end.
Resources
BYO LLM documentation
OpenAI Realtime API on Azure example
Text-based LLMs and data residency
Sign up for Voximplant
