WebSocket API Client

Ultravox is a multimodal LLM that understands text and human speech without separate ASR. It converts audio to LLM’s high-dimensional space, responding faster than systems with separate ASR and LLM components.

You can integrate Ultravox WebSocket API client in your Voximplant application via the Ultravox VoxEngine module. It provides you a client interface to communicate with Ultravox models in real time and produce both audio and text transcriptions.

1. Require the Modules.Ultravox module in your VoxEngine scenario.
2. Create an Ultravox.WebSocketAPIClient instance (via the Ultravox.createWebSocketAPIClient method) and provide your parameters.
3. Send media between Call and Ultravox.WebSocketAPIClient (via the *.sendMedia or VoxEngine.sendMediaBetween methods).
4. Listen to the Ultravox.WebSocketAPIClient events (via the Ultravox.WebSocketAPIEvents event list) and implement you application business logic.

Connecting to Ultravox agents

There are 3 ways fo connecting your Voximplant application to Ultravox agents:

1. Call to an agent is created via the Create Call API method from a Voximplant scenario.
2. Call to an agent is created via the Create Agent Call API method from a Voximplant scenario.
3. Voximplant scenario joins an existing Ultravox call via the joinUrl setting.
   Important: The Ultravox call should be created with medium set to serverWebSocket and both inputSampleRate and outputSampleRate set to 16000, for the call audio to work properly. Refer to the Ultravox documentation for more information.

Depending on the connection method, the VoxEngine scenario looks the following way:

1. Create Call method

2. Create Agent Call method

3. Existing call's joinUrl method

Q: The AI voice on the phone is choppy and high-pitched. What do I do?
A: Set the inputSampleRate property on the Ultravox side to 16000, this should fix the issue.