Audio format

ORI-Realtime speaks Opus at 24 kHz, mono, base64-encoded inside JSON events — in both directions. This differs from APIs that use raw PCM16; Opus keeps bandwidth ~10× lower, which matters for a full-duplex stream.

	Input (`input_audio_buffer.append`)	Output (`response.audio.delta`)
Codec	Opus (streaming, self-delimited)	Opus (streaming, self-delimited)
Sample rate	24,000 Hz	24,000 Hz
Channels	1 (mono)	1 (mono)
Encoding	base64 string in the `audio` field	base64 string in the `delta` field

Resample before you encode. Most microphones capture at 16 kHz or 48 kHz. Opus will happily encode audio at the wrong rate and the API will accept it — but the model will hear noise and produce empty or confused replies. If the model keeps asking “are you there?”, check your sample rate first.

Python — `sphn`

sphn is the streaming Opus codec the stack itself uses (pip install sphn). It maintains encoder/decoder state across chunks — create one writer and one reader per session and reuse them:

import sphn, base64
import numpy as np

writer = sphn.OpusStreamWriter(24000)   # one per session
reader = sphn.OpusStreamReader(24000)   # one per session

# Encode: float32 PCM in [-1, 1] at 24 kHz → base64 Opus.
# append_pcm returns the encoded bytes (may be empty while it buffers).
def encode(pcm_f32: np.ndarray) -> str | None:
    data = writer.append_pcm(pcm_f32)
    return base64.b64encode(data).decode() if data else None

# Decode: base64 Opus → float32 PCM at 24 kHz.
# append_bytes returns the decoded samples (may be empty mid-packet).
def decode(b64: str) -> np.ndarray:
    return reader.append_bytes(base64.b64decode(b64))

Resampling from a 16 kHz source:

from scipy.signal import resample_poly
pcm_24k = resample_poly(pcm_16k, up=3, down=2)

If your PCM is int16, normalize first: pcm_f32 = pcm_i16.astype(np.float32) / 32768.0.

Node.js

Use an Opus binding such as @discordjs/opus (native, fast) or opusscript (pure JS/WASM). Configure the codec for 24,000 Hz, 1 channel, and frame sizes of 1920 samples (80 ms) or 480 samples (20 ms). Base64-encode each encoded packet into the audio field, and base64-decode each delta before feeding your decoder.

Browser

Two workable paths:

WebCodecs (Chrome/Edge): AudioEncoder/AudioDecoder with codec: "opus", sampleRate: 24000, numberOfChannels: 1. Capture the mic with getUserMedia + AudioWorklet, resample to 24 kHz in the worklet if the context runs at 48 kHz.
opus-recorder (wider support): WASM Opus encoder with a configurable encoderSampleRate: 24000.

For playback, decode into an AudioBuffer and schedule chunks back-to-back on a Web Audio AudioContext — keep a small jitter buffer (~100 ms) so word boundaries don’t click.

Chunking and pacing

Send audio in 20–80 ms chunks at real-time pace. Don’t dump a whole file at once — the turn engine reads timing, not just content, to decide when you’ve finished speaking.
Keep sending silence between user utterances. A silent stream is how the engine knows the user is done; a stopped stream just looks like a stall.
Output audio is mastered slightly quieter than typical TTS APIs — apply client-side gain if you need to match other sources.

ORI-Realtime 1.5

ORI Utter

Python — `sphn`

Node.js

Browser

Chunking and pacing

​Python — sphn

​Node.js

​Browser

​Chunking and pacing

Python — `sphn`

Node.js

Browser

Chunking and pacing