Skip to main content

Connection

All events are JSON text frames. Binary frames are rejected.

Client → server events

Only two event types are accepted; anything else returns an error event (the connection stays open).

session.update

Send once after connecting, before audio. Can be sent again mid-session to change instructions.
string
A preset voice id (see voices). Unknown values fall back to the default voice.
string
The system prompt — persona, task, tone.
string
BCP-47-style language hint. en is the supported value; others are experimental.

input_audio_buffer.append

Stream microphone audio continuously — including silence. The turn engine uses the audio stream itself to detect when the user has finished speaking.
string
required
Base64-encoded Opus, 24 kHz mono. See audio.

Server → client events

Errors

Error events use one shape:
Fatal errors are followed by a WebSocket close with a matching code: Non-fatal errors (unsupported_event, invalid_json, invalid_frame) leave the connection open.

Limits

Session semantics

  • Sessions are ephemeral. A reconnect is a brand-new conversation; the model does not remember previous sessions. Keep your own transcript (response.text.done + user transcription events) if you need continuity.
  • Interruption is server-side. There is no response.cancel. When input_audio_buffer.speech_started arrives while audio is playing, cut your playback — the model has already stopped generating.
  • Keep audio flowing. Send silence when the user isn’t speaking. Gaps in the stream degrade turn detection.
  • Heartbeats. The server pings every 30 seconds; standard WebSocket clients answer automatically. No action needed.