Audio formats

The container and codec options on the speech endpoint and on realtime sessions.

Overview

response_format on POST /v1/audio/speech selects the container and codec of the returned audio.

Format	Kind	Use it when
`mp3`	Lossy	Something outside your control has to play the file.
`opus`	Lossy	Bandwidth matters more than compatibility.
`flac`	Lossless	You need an exact copy and can afford the size.
`wav`	Lossless	You are feeding another audio tool and want no decode step.
`pcm`	Raw	You are writing straight into an audio pipeline or buffer.

Speech is generated at 24 kHz, 16-bit, mono. A lossless format preserves that exactly; mp3 and opus trade a small amount of fidelity for a much smaller file.

Raw PCM

pcm returns signed 16-bit little-endian samples at 24 kHz, mono, with no container and no header.

Use pcm when you are appending to a playback buffer or piping into another process.
Use wav when you are writing a file someone will open. pcm carries no record of its sample rate; wav is the same samples with a header describing them.

Streaming

stream_format controls delivery. Streaming is available with pcm and wav.

`stream_format`	Delivery
omitted	One buffered response after generation finishes.
`audio`	Raw chunked audio as it is generated.
`sse`	`speech.audio.delta` and `speech.audio.done` events.

Use pcm when another tool re-ingests the bytes, and wav only when the far end requires a container. A streamed wav is written before its length is known, so read its duration from the audio rather than from the header.

Realtime audio

A realtime session sets its output in audio.output.format rather than response_format. The type is audio/pcm, audio/wav, or audio/mpeg, and the rate is 8000, 16000, 22050, 24000, 44100, or 48000 Hz. The default is audio/pcm at 24 kHz. Input is linear PCM16 at the rate you declare in audio.input.format.

For the session fields, see Speech to speech and Text to speech over a WebSocket.