Skip to content

SPEV: free through Oct 31. Details →

LiveKit: text to speech

Speak your agent's replies in a LiveKit Agents session with a Chorus preset voice.

Overview

Sprag provides an OpenAI compatible text-to-speech API allowing you to use LiveKit's OpenAI plugin out of the box. For TTS on LiveKit that means openai.TTS is usable by simply setting Sprag's base URL and invoking one of our models.

Basic usage

Use openai.TTS in an AgentSession, or as a standalone speech service:

import os

from livekit.agents import AgentSession
from livekit.plugins import openai

session = AgentSession(
    tts=openai.TTS(
        model="chorus",
        voice="wade",
        response_format="pcm",
        base_url="https://api.sprag.ai/v1",
        api_key=os.environ["SPRAG_API_KEY"],
    ),
    # ... stt, llm, etc.
)

Configuration

Constructor arguments

ArgumentTypeDefaultDescription
modelstr--Required parameter when used with Sprag
voicestr--A preset id for the model you pass.
response_formatstr--Required. pcm or wav.
base_urlstr--https://api.sprag.ai/v1.
api_keystrOPENAI_API_KEY env varYour Sprag API key.

See text-to-speech models for a list of available models.

Audio format

We stream audio back as it is generated, in pcm or wav. Choose pcm when the audio feeds straight into the session. It arrives as 24 kHz mono.

You can find more information about the choice between them in audio formats.

Voices

We speak in whichever preset you pass in the voice field, and each model has its own set.

curl "https://api.sprag.ai/v1/audio/voices?model=chorus" \
  -H "Authorization: Bearer $SPRAG_API_KEY"

You can find the full list in voice presets.

Realtime speech sessions

Our speech models also run as a realtime session over a WebSocket, which LiveKit reaches through openai.realtime.RealtimeModel. You send text as a chat message and we stream the spoken audio back on the same connection.

import os

from livekit.agents import llm
from livekit.plugins.openai.realtime import RealtimeModel

model = RealtimeModel(
    model="chorus",
    voice="wade",
    modalities=["audio"],
    base_url="wss://api.sprag.ai/v1/audio/speech",
    api_key=os.environ["SPRAG_API_KEY"],
)
session = model.session()

chat_ctx = llm.ChatContext.empty()
chat_ctx.add_message(role="user", content="Thanks for calling. How can I help?")
await session.update_chat_ctx(chat_ctx)

reply = await session.generate_reply()
async for message in reply.message_stream:
    async for frame in message.audio_stream:
        ...  # 24 kHz mono PCM frames

Reference