Skip to main content

CLI Mode

CLI mode lets you run the LLMRTC backend directly from the command line with environment variable configuration. This is the fastest way to get started.

Quick Start​

# Install the package
npm install @llmrtc/llmrtc-backend

# Set required environment variables
export OPENAI_API_KEY=sk-...

# Start the server
npx llmrtc-backend

The server starts on http://127.0.0.1:8787 by default.

Environment Variables​

Server Configuration​

VariableDefaultDescription
PORT8787HTTP/WebSocket port
HOST127.0.0.1Bind address
SYSTEM_PROMPT(built-in)System prompt for the assistant
HISTORY_LIMIT8Max messages in conversation history
STREAMING_TTStrueEnable streaming TTS
STREAMING_STTfalseStream mic audio to STT live for interim transcripts (needs a streaming STT provider, e.g. STT_PROVIDER=elevenlabs-scribe)

Provider Selection​

VariableDescription
LLM_PROVIDERForce LLM provider: openai, anthropic, gemini, bedrock, openrouter, zai (alias: glm), ollama, lmstudio
STT_PROVIDERForce STT provider: openai, whisper, faster-whisper, elevenlabs-scribe (aliases: elevenlabs, scribe), openai-realtime (alias: realtime)
TTS_PROVIDERForce TTS provider: openai, elevenlabs, piper

If not specified, providers are auto-detected based on available API keys.

AWS Bedrock​

VariableDescription
AWS_ACCESS_KEY_IDAWS access key (or use the default credential chain)
AWS_SECRET_ACCESS_KEYAWS secret key
AWS_REGIONAWS region (default: us-east-1)

Model Overrides​

VariableDescription
OPENAI_MODELOpenAI LLM model
ANTHROPIC_MODELAnthropic model
ANTHROPIC_PROMPT_CACHINGtrue enables Anthropic prompt caching (~90% input-cost saving on multi-turn conversations)
GOOGLE_MODELGemini model
BEDROCK_MODELBedrock model id (use us./eu. inference-profile ids)
OPENROUTER_MODELOpenRouter model (provider/model format)
ZAI_API_KEYZ.ai API key (for LLM_PROVIDER=zai)
ZAI_MODELZ.ai GLM model (default: glm-5.2)
OLLAMA_VISION_MODELLocal vision model for the vision provider (default: llava for compatibility; set qwen3-vl for the current generation)
OPENAI_STT_MODELOpenAI transcription model (default: whisper-1; try gpt-4o-mini-transcribe; gpt-realtime-whisper with STT_PROVIDER=openai-realtime)
ELEVENLABS_STT_MODELElevenLabs Scribe batch model (default: scribe_v2)
OPENAI_TTS_MODELOpenAI TTS model (default: tts-1; gpt-4o-mini-tts supports instructions)
OPENAI_TTS_VOICEOpenAI TTS voice (default: nova)
OPENAI_TTS_INSTRUCTIONSDelivery instructions for instructable TTS models (tone, pacing, persona)

Runtime​

VariableDescription
LLMRTC_SKIP_NODE_CHECKSet to 1 to bypass the Node 20+ startup check (unsupported)

API Keys​

VariableProvider
OPENAI_API_KEYOpenAI (LLM, STT, TTS)
ANTHROPIC_API_KEYAnthropic (LLM)
GOOGLE_API_KEYGoogle Gemini (LLM)
ELEVENLABS_API_KEYElevenLabs (TTS, Scribe STT)
OPENROUTER_API_KEYOpenRouter (LLM)

TURN Configuration​

VariableDescription
METERED_APP_NAMEMetered.ca app name for TURN
METERED_API_KEYMetered.ca API key for TURN
METERED_REGIONPreferred TURN region

Local Providers​

VariableDefaultDescription
LOCAL_ONLYfalseUse only local providers
OLLAMA_BASE_URLhttp://localhost:11434Ollama server URL
OLLAMA_MODELllama3.1Ollama model name
LMSTUDIO_BASE_URLhttp://localhost:1234/v1LM Studio server URL
LMSTUDIO_MODELlocal-modelLM Studio model name
FASTER_WHISPER_URLhttp://localhost:9000Faster-Whisper server URL
PIPER_URLhttp://localhost:5002Piper TTS server URL

Example Configurations​

OpenAI Stack​

export OPENAI_API_KEY=sk-...
export SYSTEM_PROMPT="You are a helpful voice assistant."
npx llmrtc-backend

Mixed Providers​

export ANTHROPIC_API_KEY=sk-ant-...
export OPENAI_API_KEY=sk-...
export ELEVENLABS_API_KEY=xi-...
export LLM_PROVIDER=anthropic
export TTS_PROVIDER=elevenlabs
npx llmrtc-backend

Local-Only Stack​

export LOCAL_ONLY=true
export OLLAMA_BASE_URL=http://localhost:11434
export OLLAMA_MODEL=llama3
export FASTER_WHISPER_URL=http://localhost:9000
export PIPER_URL=http://localhost:5002
npx llmrtc-backend

Using .env Files​

Create a .env file in your working directory:

# .env
OPENAI_API_KEY=sk-...
ELEVENLABS_API_KEY=xi-...
SYSTEM_PROMPT=You are a helpful assistant.
PORT=8787
STREAMING_TTS=true

Load it before starting:

# Using dotenv
node -r dotenv/config node_modules/.bin/llmrtc-backend

# Or with shell
source .env && npx llmrtc-backend

Health Check​

The server exposes a health endpoint:

curl http://localhost:8787/health
# {"ok":true}

Use this for load balancer health checks and monitoring.

Logging​

Logs are written to stdout. In production, redirect to a log file or log aggregator:

npx llmrtc-backend 2>&1 | tee server.log

Log levels and formats can be customized in library mode.

Process Management​

For production deployments, use a process manager:

PM2​

# ecosystem.config.js
module.exports = {
  apps: [{
    name: 'llmrtc',
    script: 'npx',
    args: 'llmrtc-backend',
    env: {
      OPENAI_API_KEY: 'sk-...',
      PORT: 8787
    }
  }]
};

# Start
pm2 start ecosystem.config.js

systemd​

# /etc/systemd/system/llmrtc.service
[Unit]
Description=LLMRTC Backend
After=network.target

[Service]
Type=simple
User=llmrtc
WorkingDirectory=/opt/llmrtc
EnvironmentFile=/opt/llmrtc/.env
ExecStart=/usr/bin/npx llmrtc-backend
Restart=always

[Install]
WantedBy=multi-user.target

Docker​

FROM node:20-slim

# Install FFmpeg for streaming TTS
RUN apt-get update && apt-get install -y ffmpeg && rm -rf /var/lib/apt/lists/*

WORKDIR /app
RUN npm install @llmrtc/llmrtc-backend

EXPOSE 8787
CMD ["npx", "llmrtc-backend"]