<!-- generated: do not edit. source: content/docs/decisions/extensions.md -->

# Sprag extensions

Sprag extensions add inline audio, visual video evidence, and media token
counts to `POST /v1/decisions`.

## Send video

Video evidence requires a model whose [catalog entry](/models) lists
`input_video` among its supported parameters.

```python
import base64
import os
from pathlib import Path

import requests

response = requests.post(
    "https://api.sprag.ai/v1/decisions",
    headers={"Authorization": f"Bearer {os.environ['SPRAG_API_KEY']}"},
    json={
        "model": os.environ["SPRAG_VIDEO_MODEL"],
        "input": "Delivery camera recording.",
        "input_video": {
            "data": base64.b64encode(Path("delivery.mp4").read_bytes()).decode("ascii"),
            "format": "mp4",
        },
        "questions": [
            {
                "type": "predicate",
                "name": "delivered",
                "instructions": "Does the video show a package being left at the door?",
            }
        ],
    },
)
response.raise_for_status()
print(response.json())
```

Illustrative response for `SPRAG_VIDEO_MODEL=example-video-model`:

```json
{
  "model": "example-video-model",
  "answers": [
    { "type": "predicate", "name": "delivered", "probability": 0.9 }
  ],
  "usage": {
    "input_tokens": 256,
    "input_tokens_details": {
      "cached_tokens": 0,
      "cache_write_tokens": 0,
      "multimodal_tokens": { "video": 128 }
    },
    "output_tokens": 0,
    "output_tokens_details": { "reasoning_tokens": 0 },
    "total_tokens": 256
  }
}
```

Set `SPRAG_VIDEO_MODEL` to the id of a model that accepts video evidence.
`input_video` attaches one video file alongside the required `input` and
`questions` fields.

| Field | Required | Type | Notes |
| --- | --- | --- | --- |
| `input_video.data` | Yes | string | Nonempty base64-encoded video file bytes. |
| `input_video.format` | Yes | string | A nonempty, free-form container or format hint, such as `mp4` or `webm`. |

`input_video` supplies visual frames. To include the soundtrack, attach its
audio separately as `input_audio` in the same request.

## Send audio

```json
{
  "model": "spev",
  "input": "Call recording attached.",
  "input_audio": { "data": "<base64 audio file bytes>", "format": "wav" },
  "questions": [
    { "type": "predicate", "instructions": "Does the caller ask to cancel?" }
  ]
}
```

`input_audio` attaches one audio file alongside `input`. Replace the example's
`data` with the file's base64-encoded bytes.

| Field | Required | Type | Notes |
| --- | --- | --- | --- |
| `input_audio.data` | Yes | string | Nonempty base64-encoded audio file bytes. |
| `input_audio.format` | Yes | string | A nonempty, free-form container or format hint, such as `wav` or `mp3`. |

The [model catalog](/models) lists `input_audio`, `input_image`, and
`input_video` for models that accept each kind of evidence. Image evidence uses
`input_image` parts inside user messages. For image fields and an example,
see [Send an image](/docs/decisions/api#send-an-image).

## Read answers

Media requests return the same structured answers as text requests. `answers`
follows question order and carries each question's `name`, or `null`.

| Question | Answer fields | Meaning |
| --- | --- | --- |
| Predicate | `probability` | Probability that the condition holds, from 0 to 1. |
| Choice | `choice`, `probabilities`, `confidence` | Selected string or boolean value and the distribution over your choices. |
| Score | `score`, `probabilities`, `confidence` | Expected zero-based level index and the distribution over your rubric. |

If the model declines a question, its answer has `type: "refusal"`. For the
question and answer fields, see [Ask a question](/docs/decisions/api#ask-a-question).

## Read token usage

`usage.input_tokens_details.multimodal_tokens` reports native input token
counts for `audio`, `image`, and `video`, when supplied in the response.
Absent modalities have no key. Text-only evidence has no media counts.

`usage.input_tokens` includes text and media tokens. `usage.total_tokens` is
`input_tokens` plus `output_tokens`. For the full response fields, see the
[Decisions API](/docs/decisions/api#response).