Audio captioning API

Overview

Captioning has no endpoint of its own. It runs on POST /v1/chat/completions, the OpenAI-compatible chat endpoint, with the clip attached as an input_audio content part.

Point any OpenAI chat client at it by changing the base URL and the model.

Request

curl https://api.sprag.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $SPRAG_API_KEY" \
  -d '{
    "model": "symphony-notes",
    "messages": [{
      "role": "user",
      "content": [
        {
          "type": "input_audio",
          "input_audio": {
            "data": "UklGRiQAAABXQVZF...",
            "format": "wav"
          }
        }
      ]
    }]
  }'

You can omit format; the container is identified from the bytes either way. WAV and MP3 reach the model as sent, any other recognized container is transcoded first, and bytes that decode as no known audio format come back as a 400.

You can also send the clip as an audio_url content part carrying a data: URL. Web URLs are rejected, so the audio always travels in the request body.

The model's input modality is audio only. It takes no text instructions and there is no prompt field, so you cannot steer what it listens for. Use response_format and structured_outputs to shape the output instead.

Clips are capped near four minutes. See models for the two limits that produce that ceiling.

Response

The description comes back as ordinary assistant message content, in the same chat completion envelope any text model returns.

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "A man in his sixties speaks slowly in a large, reverberant room..."
      }
    }
  ]
}

Structured output

If you are indexing captions rather than displaying them, use response_format and structured_outputs to turn the paragraph into fields you can store. In search and tagging pipelines, prefer this to keeping free text and parsing it later.

Generation parameters

Symphony Notes is a chat model, so it takes generation parameters rather than audio ones: max_tokens, temperature, top_p, top_k, min_p, frequency_penalty, presence_penalty, repetition_penalty, seed, stop, logprobs, top_logprobs, logit_bias, response_format, and structured_outputs.

temperature defaults to 0.0, the bottom of the range. Raise it to vary the wording between runs on the same clip.

Errors

See the errors reference for the full error shape and retry guidance.