Skip to main content
The Voice API enables real-time, bi-directional audio conversations with Datagrid AI Agents over WebSockets. Audio is streamed as base64-encoded PCM data, and the agent responds with synthesized speech in real time.

How it works

  1. Start a session — Call the REST endpoint or connect directly via WebSocket
  2. Connect — Open a WebSocket connection to the returned URL
  3. Stream audio — Send microphone audio as base64 PCM chunks; receive audio responses the same way
  4. End the session — Send a stop message, or simply close the WebSocket

Getting Started

There are two ways to start a voice session. Choose the one that fits your stack. Call POST /v1/voice to validate your request and receive a WebSocket URL with a ready-made start message. Then connect to the URL and send the message as the first frame.
POST /v1/voice depends on Redis to issue a short-lived REST-to-WebSocket handoff token. During a Redis incident, clients that can construct their own start message may use the direct WebSocket flow below with a raw API key.

Option B: Direct WebSocket

If you prefer to skip the REST call, connect directly to the WebSocket endpoint with your API key:
Then send a start message manually as the first frame:
You must send a start message within 30 seconds of connecting. If the server doesn’t receive one in time, it closes the connection with code 4000 (idle timeout).

Client → Server Messages

All messages are JSON objects with a type field and an optional payload.

start — Begin a voice session

audio — Send an audio chunk

Audio should be sent as 16-bit mono PCM at 16kHz, base64-encoded. Wait for the ready message before sending audio.

stop — End the session

Gracefully ends the session. The server responds with an ended message containing the session transcript and credits consumed.
stop is optional. Closing the WebSocket connection also gracefully ends the session and commits all buffered content server-side. The only difference is that with stop, you receive the ended response containing the final transcript and credit usage before the connection closes.

interrupt — Interrupt the agent

Send this when the user starts speaking while the agent is responding. The agent will stop its current response and the server sends an interrupted message.

Voice Configuration

The voice_config option in the start message allows you to customize voice session behavior:

Available Voice Presets

Server → Client Messages

started — Session established

Sent immediately after a start message is processed. Contains the IDs for the session, conversation, and initial message.

ready — Agent is ready to receive audio

Wait for this message before sending audio chunks. The agent needs a moment to initialize after the session starts.

audio — Agent audio response

Response audio is 16-bit mono PCM at 24kHz. Multiple audio messages are sent in sequence as the agent speaks.

tool_call — Agent is using a tool

Status is either "started" or "completed". Use this to show loading indicators while the agent searches knowledge or uses other tools.

transcript — Real-time transcription

Sent in real-time as transcription becomes available. The role field is either "user" or "agent". Use this to display a live transcript as the conversation progresses.

citation — Source citation

Sent when the agent references a knowledge source. Each source is a superset of the SSE /v1/converse CitationSource schema: the core fields (type, source_name, optional source_id/source_uri, and confirmations) are identical, so external clients can treat voice and SSE citations the same way. Sources also include optional per-type enrichment fields (e.g. page_number/data_lake_item_id/item_type for pdf_page, table_id/content/record_title/status/fields/item_type for record, thumbnail_url for web_search, query_view/task_explanation for sql_query_result, fact_data for action) that clients may use or ignore. The timestamp_ms is relative to the session start, and message_id is the stable id of the agent turn that produced the citations — use it to correlate a citation to the turn it supports.

interrupted — Agent was interrupted

Confirms that the agent’s response was interrupted after a client interrupt message.

error — An error occurred

Errors do not necessarily close the session. Transient errors are recoverable — only fatal errors are followed by a WebSocket close.

ended — Session ended

Sent when the session ends (either from a stop message, server-side timeout, or error). Contains the final transcript and credit usage.

Session Lifecycle

WebSocket Close Codes

Voice Orchestrator Tasks

Default orchestrator voice sessions can delegate longer-running work to specialist agents. When delegated work continues after the voice-safe turn budget, the server persists task status so clients can show a user-scoped task inbox. Use these REST endpoints to surface delegated task state: Supported task states are queued, running, completed, failed, and cancelled. The cancelled state is reserved for terminal records produced by future cancellation flows; this API does not currently expose a cancel operation. Missing, unowned, expired, non-terminal acknowledgement, and feature-disabled tasks are returned as not found. The current backend persists task status and terminal results. In-flight specialist execution still runs in the existing voice server process, so a server crash or redeploy during execution can leave a task running until its expires_at time.

Audio Format Reference

Platform Guides