Audio captioning
Describe what a recording contains, including speakers, emotion, music, and non-speech sound.
Captioning describes a recording, including everything in it that is not speech. Send a clip to Symphony Notes and it returns a plain-language paragraph covering what was said, who said it, how they sounded, and the sound around them.
| You want | Use |
|---|---|
| The words, with timings and speakers | Speech to text |
| A description of the whole recording | Captioning |
Use captioning when the sound matters as much as the speech: a buzzer, a door, a change in the music, a speaker whose tone contradicts their words.
Get an API key
Create a Sprag account and open API keys in the app to generate one.
Copy the key immediately. The full value is shown once and is not recoverable after you leave the page.
Export it as an environment variable, replacing the placeholder with your key:
export SPRAG_API_KEY=<your_sprag_api_key>Make a request
Send the clip to the chat endpoint as an input_audio content part. The model
reads audio only, so the message carries no text.
curl https://api.sprag.ai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $SPRAG_API_KEY" \
-d '{
"model": "symphony-notes",
"messages": [{
"role": "user",
"content": [
{
"type": "input_audio",
"input_audio": { "data": "UklGRiQAAABXQVZF...", "format": "wav" }
}
]
}]
}'Read the description
The caption comes back as ordinary assistant message content.
{
"choices": [
{
"message": {
"role": "assistant",
"content": "A man in his sixties speaks slowly in a large, reverberant room..."
}
}
]
}Clips are capped near four minutes, and a clip past the cap fails rather than thinning the description. Split long recordings and caption each piece. See models for the two limits behind the ceiling.
What to build with it
Accessibility. Generate audio descriptions and alt-text for sound, at catalog scale.
Media search. Make an audio or video archive findable by what is actually in each clip, not just by whatever metadata someone typed.
Dataset labeling. Use captions as labels for audio training data.
Moderation and review. Catch tone, distress, or a background event that a transcript misses entirely.
Where to go next
API. Look up the request shape, generation parameters, and structured output for indexing.
Models. See what Symphony Notes hears, the clip limits, and how it differs from the transcription models.