Audio captioning models
Symphony Notes, what it hears, and how it differs from the transcription models.
One model captions audio: Symphony Notes, on POST /v1/chat/completions. Send
the id hyphenated as symphony-notes. The model field is case-insensitive,
but the spaced name does not resolve.
Symphony Notes
A general-purpose audio captioner. It hears speech, speakers, and background sound together and returns a plain-language description, not a transcript.
| Field | Value |
|---|---|
| Model value | symphony-notes |
| Modalities | Audio in, text out |
| Endpoint | POST /v1/chat/completions |
| Input | One clip as an input_audio content part |
What it hears
- Speech content. What was said, described rather than transcribed word for word.
- Speakers and emotion. Who is talking, and the feeling in how they say it.
- Delivery. Pace, volume, and the way a line is carried.
- Acoustic environment. The room, the street, the hall behind the voice.
- Non-speech sound. Music, a buzzer, footsteps, a door.
Caption content varies with the clip, so don't parse it for expected fields. To get stable fields, request structured output.
Clip limits
Two limits cap a clip near four minutes.
| Limit | Value | Audio it allows |
|---|---|---|
| Context | 4096 tokens, audio at about 13 tokens per second | ~5m15s, before the caption takes its share |
| Media size | 8 MiB, measured after transcoding | ~4m22s |
Over the media limit the request returns 413; over the context window it
cannot run at all. Neither limit degrades gracefully, so segment long recordings
and caption each piece. Shorter clips generally get more detailed descriptions,
and per-segment captions index against timecodes.
How it compares
Several models take audio in. Pick by what you want back.
| Model | Returns | Use it when |
|---|---|---|
| Symphony Notes | Prose describing the recording | The sounds matter as much as the words |
| Rhythm | A transcript, with timings and speakers | You need the words themselves |
| Symphony | A spoken or written reply | You want the model to answer |
Symphony Notes and Rhythm both take audio and return text. Use Symphony Notes for a description of the recording: asked for a transcript, it returns prose about the speaking. Use Rhythm for the words themselves: it reports nothing about the music.
See the model catalog for the live list and pricing for current rates.