LiveKit: speech to text

Transcribe user audio in a LiveKit Agents session with Sprag models.

Overview

LiveKit's openai.STT plugin works with Sprag's transcription models. Set the Sprag base URL, name a model, and use it in an AgentSession or as a standalone transcription service.

Basic usage

import os

from livekit.agents import AgentSession
from livekit.plugins import openai

session = AgentSession(
    stt=openai.STT(
        model="rhythm",
        use_realtime=True,
        base_url="https://api.sprag.ai/v1",
        api_key=os.environ["SPRAG_API_KEY"],
    ),
    # ... llm, tts, etc.
)

Configuration

Constructor arguments

Argument	Type	Default	Description
`model`	`str`	--	A Sprag transcription model id. Required.
`use_realtime`	`bool`	`False`	Stream over a realtime WebSocket instead of one request per turn.
`base_url`	`str`	--	`https://api.sprag.ai/v1`.
`api_key`	`str`	`OPENAI_API_KEY` env var	Your Sprag API key.
`turn_detection`	`dict`	server VAD	Endpointing on the realtime transport.

See speech-to-text models for the available models and supported languages for coverage.

Transports

With use_realtime=True, the plugin holds one WebSocket open across the session and streams audio as the user speaks. With False, each turn is posted as a separate request once it ends.

VAD

On the realtime transport, Sprag's server-side VAD finds turn boundaries for you. Configure it through turn_detection, which takes the same fields as the realtime API; any field you leave out keeps its default. It applies with use_realtime=True only.

import os

from livekit.agents import AgentSession
from livekit.plugins import openai

session = AgentSession(
    stt=openai.STT(
        model="rhythm",
        use_realtime=True,
        base_url="https://api.sprag.ai/v1",
        api_key=os.environ["SPRAG_API_KEY"],
        turn_detection={"silence_duration_ms": 200},
    ),
)

You can also run VAD through LiveKit's own plugins.

Events

A realtime session reports the edges of each turn alongside its transcript.

Event	Meaning
`START_OF_SPEECH`	The turn started.
`INTERIM_TRANSCRIPT`	A fragment of the turn's transcript.
`FINAL_TRANSCRIPT`	The settled transcript for that turn.
`END_OF_SPEECH`	The turn ended.
`RECOGNITION_USAGE`	Measured duration for that turn.

Fragments concatenate into the final transcript, and arrive once the turn closes. Over REST you receive one FINAL_TRANSCRIPT per utterance.

Speaker labels and word timings

For word timings or speaker labels, post the utterance to POST /v1/audio/transcriptions with response_format=verbose_json and, for speakers, speaker_labels=true. See timestamps and diarization.