Audio formats

The container and codec options on the speech endpoint, and the raw PCM the realtime session uses.

Overview

response_format on POST /v1/audio/speech selects the container and codec of the returned audio. The realtime API does not use it; that path is raw PCM only.

Format	Kind	Use it when
`mp3`	Lossy	Something outside your control has to play the file.
`opus`	Lossy	Bandwidth matters more than compatibility.
`flac`	Lossless	You need an exact copy and can afford the size.
`wav`	Lossless	You are feeding another audio tool and want no decode step.
`pcm`	Raw	You are writing straight into an audio pipeline or buffer.

Those five are the whole enum, and any other value is a 400. Omit the field and the model's own container comes back.

The model generates at 24 kHz, 16-bit, and every format above is derived from that. A lossless format preserves what the model produced; it does not improve on it. mp3 and opus re-encode, adding a generational loss that is generally inaudible for speech at this rate in exchange for a large size saving.

Raw PCM

pcm returns signed 16-bit little-endian samples at 24 kHz, mono, with no container and no header. The first byte of the body is the first sample.

Use pcm when you are appending to a playback buffer or piping into another process.
Use wav when you are writing a file someone will open. Nothing in pcm records the sample rate; wav is the same samples with a header describing them.

Streaming

stream_format controls delivery, and it also narrows encoding: pcm and wav are the only formats that stream, and any other response_format sent alongside it is a 400. With stream_format set and response_format omitted, you get wav.

`stream_format`	Delivery
omitted	One buffered response after generation finishes.
`audio`	Raw chunked audio as it is generated.
`sse`	`speech.audio.delta` and `speech.audio.done` events.

Use pcm when another tool re-ingests the bytes, and wav only when the far end requires a container. Streamed wav carries 0xFFFFFFFF in the RIFF size fields, since the length is unknown when the header goes out. Players cope with that, but anything reading duration or size from the header misreports it.

Realtime audio

The realtime API does not take response_format. Sessions carry linear PCM16 in both directions, and the output side is fixed at 24 kHz: a rate declared under audio.output.format is discarded, and session.updated echoes 24000 back. Declare the input format instead.

{
  "audio": {
    "input": { "format": { "type": "audio/pcm", "rate": 16000 } }
  }
}

pcm16 and audio/pcm are both recognized for the type; use the object form, which is what carries a rate. G.711 is refused in both recognized spellings, the bare g711_ulaw and g711_alaw strings and the object types audio/pcmu and audio/pcma, with unsupported_audio_format; the whole session.update is dropped rather than partly applied. An unrecognized type is ignored silently, so read session.updated to confirm what took. Transcode telephony audio before the session rather than declaring the telephony codec.