Pipecat

Pipeline stages

Pipecat is a framework for building voice-enabled, real-time, multimodal AI applications. A typical Pipecat pipeline for voice applications looks like this:

1. Send audio. Transmit and capture streamed audio from the user.
2. Transcribe speech. Convert speech to text as the user is talking.
3. Process with an LLM. Generate a response.
4. Convert to speech. Turn the text response into natural speech.
5. Play audio. Stream the audio response back to the user.

Sprag covers steps 2 through 4 with Pipecat's own OpenAI services, so there is no Sprag-specific plugin to install. For transcription, change the base URL. For speech, change the base URL and add two lines of setup, because in Pipecat 1.8.1 OpenAITTSService checks the requested voice against a built-in list of OpenAI voice names before it makes any HTTP call. A single realtime service can also cover steps 2 through 4 at once; see realtime LLM below.

Installation

Install the OpenAI extras for Pipecat:

uv add "pipecat-ai[openai]"

Then set your Sprag API key as an environment variable:

export SPRAG_API_KEY=<your_sprag_api_key>

Generate a key from API keys in the Sprag app. See the quickstart if you do not have an account yet.

Connection

Pass base_url and api_key to the service constructor. They are constructor arguments on every service, not fields of its Settings.

The exceptions are OpenAIRealtimeSTTService and OpenAIRealtimeLLMService, which take the WebSocket base URL wss://api.sprag.ai/v1/realtime.

Services

Speech to text. OpenAISTTService posts one request per user turn, OpenAIRealtimeSTTService streams a transcript over a WebSocket. Both run on Rhythm or Symphony.

Text to speech. OpenAITTSService speaks the reply with a Chorus preset voice.

Realtime LLM. OpenAIRealtimeLLMService replaces the transcribe, process, and synthesize stages with one WebSocket.

Choosing between them

Separate transcription, LLM, and synthesis stages let you pick the LLM independently of the transcription and synthesis models. OpenAIRealtimeLLMService collapses those three stages into one hop, but the bot is still a Pipecat Pipeline -- the same transport classes, Frame types, and event handlers as any other service. Sprag's realtime API is the same one-hop session with neither: no Pipecat transport, no frame pipeline, just the WebSocket and whatever audio capture and playback you write yourself.

Use separate stages to pick the reasoning model independently. Use Pipecat's realtime service for one hop while keeping Pipecat's transports and pipeline. Use the realtime API directly when you are not building on Pipecat at all.