Websocket-based Voice AI Telephony Integration

​

Overview

• Outbound calls: sales, reminders, scheduling, comfirmations, surveys
• Inbound calls: service, support
• Bi-directional

​

Setup

1. Have your agent ready and running on a publically accessible WebSocket endpoint.
2. Have an Awaaz AI account with:
   • A template set up for calls
   • An Indian phone number assigned
   • The WebSocket URL configured

​

How to Use

​

1. Upload Messages

• Log in to the Awaaz AI Portal UI
• Upload the messages you want to deliver via the XFin API
• Awaaz AI scheduler will queue up the messages for sending calls

​

2. Outgoing Calls

• Calls are triggered to end users via your Indian phone number.
• During each call, Awaaz AI opens a WebSocket connection to your agent.

​

3. Real-Time Audio Exchange

• Once connected:
  • User → Awaaz AI → Agent (user’s audio is sent as encoded JSON messages).
  • Agent → Awaaz AI → User (agent’s audio is returned the same way).

​

Message Formats

​

Messages from Awaaz AI

1. Start Message:

• This message will be sent directly after websocket connection is established.
• It will contain audio formating information and any data that might be required by the agent for executing call.

{
  "event": "start",
  "sequence_number": 1,
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "start": {
    "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "call_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "media_format": {
        "encoding": "base64",
        "sample_rate": 8000,
        "bit_rate": "" },
    "custom_parameters": {
      "FirstName": "Jane",
      "LastName": "Doe",
      "RemoteParty": "Bob",
    }
  }
}

2. Media Message:

• AwaazAI will continuously send call audio in below json encoded message format.
• Each payload will contain 320 bytes of base64 encoded, 16-bit PCM, 8 kHz audio.

{
  "event": "media",
  "sequence_number": 2,
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "media": {
    "chunk": 1,
    "timestamp": "5",
    "payload": "<>"
  } 
}

3. DTMF Message:

• This message will be sent when a digit is pressed in call by user.

{
 "event": "dtmf",
 "stream_sid":"MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
 "sequence_number":3,
 "dtmf": {
     "digit": "<>",
     "duration":"<duration in ms>"
 }
}

4. Mark Message:

• This message is used to notify when media processing is completed

{
  "event": "mark",
  "sequence_number": "4",
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "mark": {
    "name": "my label"
  }
}

5. Stop Message:

• This message will be sent when call has ended.

{
  "event": "stop",
  "sequence_number": 5,
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "stop": {
    "call_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  }
}

​

Messages to Awaaz AI

1. Media Message:

• This message will be received from your websocket with audio to be played in call

{
  "event": "media",
  "sequence_number": 1,
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "media": {
    "chunk": 1,
    "timestamp": "5",
    "payload": "<>"
  } 
}

2. Mark Message:

• This message is sent by your websocket to mark a media point. Once audio is processed, you will recieve mark event message from Awaaz AI with matching name

{
  "event": "mark",
  "sequence_number": 2,
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "mark": {
    "name": "my label"
  }
}

3. Clear Message:

• Send this message to clear audio that has been sent but not played yet.

{
  "event": "clear",
  "stream_sid": "MZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}

​

Example Client Code for Your Application

1. Receiving audio chunks in JSON messages.
2. Accumulating a certain number of chunks.
3. Sends audio from an audio file in chunks

Note: This example is intended as a reference. You should refactor it to fit the specific requirements of your agent or application.