Speech to text in realtime

Realtime transcription lets you transcribe a live stream without cutting it into files first. You open one WebSocket for the whole conversation, stream audio as it is captured, and the server finds the turn boundaries and returns each turn's transcript as OpenAI-compatible input-transcription events.

A turn is transcribed once it closes, so its transcript arrives after the input_audio_buffer.speech_stopped event rather than word by word.

This page covers connecting, declaring the model, sending audio, and reading the events that come back. Models lists the models that serve transcription sessions; the examples below use rhythm. For a session that also speaks a reply, see speech to speech. For text in another language, see Translate a live stream.

Connection

Sessions run over WebSocket at the transcription route:

wss://api.sprag.ai/v1/audio/transcriptions

A session on this route is a transcription session. The shared realtime route opens the same session when intent=transcription is on the URL:

wss://api.sprag.ai/v1/realtime?intent=transcription

Authentication

Authenticate the upgrade with your API key in the bearer header:

Authorization: Bearer $SPRAG_API_KEY

A client that cannot set headers on a WebSocket upgrade, such as the browser WebSocket API, passes the key as a subprotocol pair instead: sprag-api-key followed by the key.

new WebSocket("wss://api.sprag.ai/v1/audio/transcriptions", [
  "sprag-api-key",
  spragApiKey,
]);

If your API key is missing or rejected, the WebSocket upgrade fails with HTTP 401 before the socket opens.

Declaring the model

A transcription session starts with no model. Within ten seconds of connecting, send a session.update that names one under session.audio.input.transcription.model. The same update can carry your input format and turn-detection settings:

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "format": { "type": "audio/pcm", "rate": 16000 },
        "transcription": { "model": "rhythm" },
        "turn_detection": {
          "type": "server_vad",
          "threshold": 0.5,
          "prefix_padding_ms": 300,
          "silence_duration_ms": 500
        }
      }
    }
  }
}

The server emits session.created on connect and answers each accepted update with session.updated, whose payload is the resolved session.

Audio format

Input is linear PCM16. Declare your capture rate in audio.input.format, as an object of the form {"type": "audio/pcm", "rate": 16000}. Transcode other codecs, including G.711 telephony audio, to PCM16 before streaming.

Send audio base64-encoded in input_audio_buffer.append events:

{ "type": "input_audio_buffer.append", "audio": "<base64 pcm16>" }

Turn detection

Server VAD closes the user turn when input audio goes quiet for the configured duration. Every field is optional.

To mark turns yourself, set turn_detection to null and close each turn with input_audio_buffer.commit. A commit needs at least 100 ms of audio.

Events

Deltas and the completed event both carry an item_id, so accumulate text per item. A completed event looks like this:

{
  "type": "conversation.item.input_audio_transcription.completed",
  "item_id": "item_cd99c2b70002",
  "content_index": 0,
  "transcript": "Thanks, everyone, for joining.",
  "usage": { "type": "duration", "seconds": 1.696 }
}

For word-level timings, use REST; see timestamps.

Limits

If your organization is at its cap of concurrent sessions, the server accepts a new transcription session and then closes it with close code 1013. More information about rate limits can be found in the API docs.

Browsers and most WebSocket libraries answer pings automatically. If yours answers only while a receive is pending, keep receiving between turns.

Close codes