Speech-to-text models
The models that turn audio into text, and which one to pick for transcription, conversation, or description.
Three models take audio in and return text; Symphony can also reply with speech.
| Model | Model ID | Endpoint | Use it for |
|---|---|---|---|
| Rhythm | rhythm | /v1/audio/transcriptions | Writing down the words that were said |
| Symphony | symphony | /v1/audio/transcriptions, /v1/chat/completions, or a realtime session | Writing down the words, or replying to them |
| Symphony Notes | symphony-notes | /v1/chat/completions | Describing what is happening in the audio |
Both Rhythm and Symphony are transcription capable. Rhythm is a standard ASR model where Symphony particularly excels in multilingual tasks or more complex auditory environments.
Pass the id in the model field; the names are for reading.
Rhythm
The transcription model. It returns the transcript only, with timestamps and diarization as structured fields.
| Field | Value |
|---|---|
| Model ID | rhythm |
| Modalities | Audio in, text out |
| Endpoint | POST /v1/audio/transcriptions |
| Parameters | language, response_format, timestamp_granularities[], speaker_labels, num_speakers |
| Languages | Thirty, plus twenty-two Chinese dialects. See languages. |
REST has the request contract. A realtime transcription session runs the same model over a WebSocket.
Symphony
A multimodal model that takes text, image, audio, and video in and returns text
and speech. It transcribes on /v1/audio/transcriptions like Rhythm, answers on
/v1/chat/completions, and in a realtime session hears the
user and replies out loud.
Use Symphony when you want a reply rather than a transcript, or when one model should do both. It does the hearing and the answering together, instead of transcribing and then prompting separately. Or when you need speech output and a transcript in a single request.
| Field | Value |
|---|---|
| Model ID | symphony |
| Modalities | Text, image, audio, video in; text and audio out |
| Endpoint | POST /v1/audio/transcriptions, POST /v1/chat/completions, or a realtime session |
| Speech understanding | Eighteen named languages, seventeen of them shared with Rhythm |
On the chat endpoint it takes generation parameters rather than transcription
ones: max_tokens,
temperature, top_p, top_k, min_p, frequency_penalty,
presence_penalty, repetition_penalty, seed, stop, logit_bias,
response_format, and structured_outputs.
Symphony Notes
Audio captioning. It describes a clip end to end: speech content, speakers, emotion, delivery, and the acoustic environment, including non-speech sound.
It returns prose about the audio rather than the words in it, so use it to describe a recording and use Rhythm to transcribe one. Audio captioning covers it in full.
| Field | Value |
|---|---|
| Model ID | symphony-notes |
| Modalities | Audio in, text out |
| Endpoint | POST /v1/chat/completions |
The model catalog carries the live list, and pricing has the current rates.