Connecting Dialogflow agent to outbound calls

Connecting Dialogflow agent to outbound calls


Please note that this tutorial is about connecting Dialogflow agent to outbound calls, If you need to connect inbound calls to your Dialogflow agent please follow the "How To Use Dialogflow Connector" tutorial. Recently Google has introduced Google Duplex, it's not based on Dialogflow, but you still can build something similar using Dialogflow Connector and Voximplant.

Outbound call connected to Dialogflow

Dialogflow Connector lets developers connect a call running via Voximplant to a Dialogflow agent, which will do speech recognition, natural language processing (NLP) and speech synthesis according to its logic. Audio streaming from Voximplant to and from Dialogflow happens in real-time, query results arrive to VoxEngine as soon as they are returned by an agent. 

It's required that Dialogflow agent should be set up to support API V2 (since API V1 doesn't support streaming) and Beta features should be enabled (Settings -> Enable beta features and APIs) since speech synthesis is in the beta features list at the moment.

Setting up Dialogflow

Re-check that your Dialogflow agent uses API V2. Now create and download the service account JSON file associated with the agent from GCP console (please read the Setting up authentication article for details). "Dialogflow API Client" role should be chosen while creating the service account. The JSON file is required for Voximplant to be authorized before sending audio data to the agent.

Click on the Speech tab in the agent settings to setup Speech synthesis options.

Speech Synthesis Settings

Enable Automatic Text to Speech by clicking on the toggle button, choose MP3 or OGG in the Output Audio Encoding dropdown (IMPORTANT: only MP3 and OGG are currently supported) and one of the available voices - we highly recommend to use WaveNet-powered voices, since they sound much better than standard ones. Save settings by clicking the Save button in the top right corner.

Setting up Voximplant

 In Voximplant control panel go to your application and switch to the Dialogflow connector tab. Here, click Add in the center of the screen or Add Dialogflow agent in the upper right corner.

The Add Dialogflow agent dialog will appear, click the Choose a file button or drag'n'drop the JSON file downloaded from GCP console and click the Add button. After the file is uploaded you will see a newly added agent in the list:

You can also use visit Marketplace in the main menu to deploy an application and a test phone number that will be connected to your Dialogflow agent – to do so, click Install in the appropriate tile. Then follow the instructions to do a test call and be connected to the Dialogflow agent.

IMPORTANT: If something went wrong it could be required to delete the test application in the Applications section and the test scenario in the Scenarios section of the control panel before you try to create the test application again. Their names start with DF_ (for scenarios) or df- (for applications). Please note that default test application can work with an agent that supports English language, if your Dialogflow agent uses another language please change the test Voxengine scenario accordingly. 

These first steps are the same for inbound and outbound calls, now comes the part explaining how to connect your agent to outbound calls

Setting up Voximplant – Buying real phone number

To make calls to phone network from Voximplant real phone number is required, since it will be used as a caller ID. On the Voximplant control panel, select Number on the left menu and then select Buy new phone number in the upper left corner.

The phone number interface will be displayed. Select Test numbers, select one or more virtual numbers and click Buy selected.

After you’ve bought a number, you'll immediately see this number on the list.

To associate a phone number with an application, go to Applicationsclick on the test application that we have previously created and switch to the Numbers tab. Here, click on the Available tab, select a number and click Attach:

If you've chosen the number in the country where no special verification is required (for example, in The United States) then you will be able to make calls now, otherwise there will be a requirement to upload verification documents required by authorities in the country. It will take some time until the number becomes active if all the submitted verification documents are correct.

Setting up Voximplant - Modifying scenario for outbound calls

The test application creation process includes the following actions: Voximplant application is created, Voxengine scenario is created, then test phone number is bought and the scenario connected to the application by creating the application Rule that forwards all calls from the test number to the scenario. In the end service account JSON file is bound to the application. By default, the Voxengine scenario connects inbound calls to a Dialogflow agent, but we need to change it to work with outbound calls:


var dialogflow, call, hangup

// Inbound call processing - remove processing by commenting the code
VoxEngine.addEventListener(AppEvents.CallAlerting, (e) => {
	call =
	call.addEventListener(CallEvents.Connected, onCallConnected)
	call.addEventListener(CallEvents.Disconnected, VoxEngine.terminate)

// Create outbound call as soon as StartScenarios HTTP API arrives
VoxEngine.addEventListener(AppEvents.Started, (e) => {
     let number = VoxEngine.customData() // we assume that a callee's number arrives as customData in e.164 format
     call = VoxEngine.callPSTN(number, CALLER_ID) // replace CALLER_ID with the number we bought in previous step (the real one, test number won't work)
     call.addEventListener(CallEvents.Connected, onCallConnected)
     call.addEventListener(CallEvents.Disconnected, VoxEngine.terminate)
     call.addEventListener(CallEvents.Failed, VoxEngine.terminate)

function onCallConnected(e) {
  // Create Dialogflow object
	dialogflow = AI.createDialogflow({
	  lang: DialogflowLanguage.ENGLISH_US
	dialogflow.addEventListener(AI.Events.DialogflowResponse, onDialogflowResponse)
    // Sending WELCOME event to let the agent says a welcome message
    dialogflow.sendQuery({event : {name: "WELCOME", language_code:"en"}})
    // Playback marker used for better user experience
    // Start sending media from Dialogflow to the call
    dialogflow.addEventListener(AI.Events.DialogflowPlaybackFinished, (e) => {
      // Dialogflow TTS playback finished. Hangup the call if hangup flag was set to true
      if (hangup) call.hangup()
    dialogflow.addEventListener(AI.Events.DialogflowPlaybackStarted, (e) => {
      // Dialogflow TTS playback started
    dialogflow.addEventListener(AI.Events.DialogflowPlaybackMarkerReached, (e) => {
      // Playback marker reached - start sending audio from the call to Dialogflow

// Handle Dialogflow responses
function onDialogflowResponse(e) {
  // If DialogflowResponse with queryResult received - the call stops sending media to Dialogflow
  // in case of response with queryResult but without responseId we can continue sending media to dialogflow
  if (e.response.queryResult !== undefined && e.response.responseId === undefined) {
  } else if (e.response.queryResult !== undefined && e.response.responseId !== undefined) {
  	// Do whatever required with e.response.queryResult or e.response.webhookStatus
        // If we need to hangup because end of conversation has been reached
        if (e.response.queryResult.diagnosticInfo !== undefined && 
           e.response.queryResult.diagnosticInfo.end_conversation == true) {
           hangup = true

    // Telephony messages arrive in fulfillmentMessages array
    if (e.response.queryResult.fulfillmentMessages != undefined) {
    	e.response.queryResult.fulfillmentMessages.forEach((msg) => {
      		if (msg.platform !== undefined && msg.platform === "TELEPHONY") processTelephonyMessage(msg)

// Process telephony messages from Dialogflow
function processTelephonyMessage(msg) {
  // Transfer call to msg.telephonyTransferCall.phoneNumber
  if (msg.telephonyTransferCall !== undefined) {
    * Example:
    * dialogflow.stop()
    * let newcall = VoxEngine.callPSTN(msg.telephonyTransferCall.phoneNumber, "put verified CALLER_ID here")
    * VoxEngine.easyProcess(call, newcall)
  // Synthesize speech from msg.telephonySynthesizeSpeech.text
  if (msg.telephonySynthesizeSpeech !== undefined) {
    // See the list of available TTS languages at
    // Example: 
    // if (msg.telephonySynthesizeSpeech.ssml !== undefined) call.say(msg.telephonySynthesizeSpeech.ssml, {"language": VoiceList.Amazon.en_US_Joanna})
    // else call.say(msg.telephonySynthesizeSpeech.text, {"language": VoiceList.Amazon.en_US_Joanna})
  // Play audio file located at msg.telephonyPlayAudio.audioUri
  if (msg.telephonyPlayAudio !== undefined) {
    // audioUri contains Google Storage URI (gs://), we need to transform it to URL (https://)
    let url = msg.telephonyPlayAudio.audioUri.replace("gs://", "")
    // Example: call.startPlayback(url)

After you fixed the scenario, click the Save button and switch to Routing on the left menu and click Run rule:

The "Run rule" dialog will be displayed. In the dialog specify a callee's phone number in Script Custom Data and click the Run rule button to initiate a call:

If everything was done correctly, the call will arrive to the specified phone number and you will hear a greeting from your agent.

Launching the scenario using API

If you want to initiate a call programmatically you need to use the StartScenarios method of HTTP API. You can also use Voximplan't CallLists if you need to initiate many calls.

Tags:dialogflowaimachine learningvoice botduplex
B6A24216-9891-45D1-9D1D-E7359CEB8282 Created with sketchtool.


Add your comment

Please complete this field.


Sign up for a free Voximplant developer account or talk to our experts