Troubleshooting

This guide covers common issues, error codes, and debugging techniques for LLMRTC applications.

Error Codes Reference

When errors occur, the server sends structured error messages with these codes:

Connection Errors

Code	Description	Common Causes
`WEBRTC_UNAVAILABLE`	WebRTC not supported or blocked	Browser incompatibility, HTTPS required
`CONNECTION_FAILED`	Connection establishment failed	Network issues, firewall blocking UDP
`SESSION_NOT_FOUND`	Session ID not recognized	Reconnecting to expired session
`SESSION_EXPIRED`	Session timed out	Inactivity beyond TTL (default 30 min)

Provider Errors

Code	Description	Common Causes
`STT_ERROR`	Speech-to-text failed	Invalid audio, provider API error
`STT_TIMEOUT`	STT processing exceeded timeout	Audio too long, slow provider
`LLM_ERROR`	LLM inference failed	Invalid prompt, API key issues
`LLM_TIMEOUT`	LLM response exceeded timeout	Complex query, provider overload
`TTS_ERROR`	Text-to-speech synthesis failed	Invalid text, provider API error
`TTS_TIMEOUT`	TTS exceeded timeout	Long text, slow provider

Processing Errors

Code	Description	Common Causes
`AUDIO_PROCESSING_ERROR`	Audio processing failed	Corrupted audio, format mismatch
`VAD_ERROR`	Voice activity detection failed	Invalid audio format
`INVALID_MESSAGE`	Malformed protocol message	Client/server version mismatch
`INVALID_AUDIO_FORMAT`	Unsupported audio format	Wrong sample rate, encoding

Playbook/Tool Errors

Code	Description	Common Causes
`TOOL_ERROR`	Tool execution failed	Tool threw exception, invalid arguments
`PLAYBOOK_ERROR`	Playbook orchestration failed	Invalid stage, missing handler

Generic Errors

Code	Description	Common Causes
`INTERNAL_ERROR`	Unexpected server error	Bug, resource exhaustion
`RATE_LIMITED`	Too many requests	Provider rate limit hit

Common Issues

No Audio / Microphone Blocked

Symptoms:

Browser console shows NotAllowedError: Permission denied
No speechStart events triggered
Microphone icon not appearing in browser

Solutions:

Ensure the page is served over HTTPS or localhost
Check browser permission settings for the site

Verify getUserMedia is called correctly:

const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    echoCancellation: true,
    noiseSuppression: true
  }
});

WebRTC Connection Fails

Symptoms:

Client stuck in connecting state
Browser console shows ICE connection failed
Works on localhost but not in production

Log excerpt:

ICE connection state: failed
ICE gathering state: complete
No valid ICE candidates found

Solutions:

Add TURN servers - Required for users behind symmetric NAT:

const server = new LLMRTCServer({
  metered: {
    appName: 'your-app',
    apiKey: process.env.METERED_API_KEY!
  }
});

Check firewall rules - Allow UDP on ports 3478, 5349, and 49152-65535

Verify signalling URL matches the server:

// Client
const client = new LLMRTCWebClient({
  signallingUrl: 'wss://your-server.com'  // Use wss:// for production
});

Test with STUN only to isolate issues:

iceServers: [{ urls: 'stun:stun.l.google.com:19302' }]

High Latency

Symptoms:

Long delay between speaking and response
Turn-around time > 2 seconds

Diagnosis checklist:

// Add timing hooks to identify bottleneck
hooks: {
  onSTTEnd: (ctx, result, timing) => {
    console.log(`STT: ${timing.durationMs}ms`);
  },
  onLLMEnd: (ctx, result, timing) => {
    console.log(`LLM: ${timing.durationMs}ms`);
  },
  onTTSEnd: (ctx, timing) => {
    console.log(`TTS: ${timing.durationMs}ms`);
  }
}

Solutions by component:

Component	Solution
STT slow	Use `whisper-1` model, ensure audio is short
LLM slow (high TTFT)	Use `gpt-5.6-luna`, `gpt-5.6-terra`, or `gemini-3.5-flash`; reduce system prompt
LLM slow (streaming)	Enable streaming (default)
TTS slow	Enable `streamingTTS: true`; use shorter responses
Network	Deploy backend closer to users; use edge regions

TTS Produces Silence

Symptoms:

ttsComplete event fires but no audio plays
Works in development but not production

Log excerpt:

Error: FFmpeg not found
TTS streaming disabled, falling back to non-streaming

Solutions:

Install FFmpeg (required for streaming TTS):

# macOS
brew install ffmpeg

# Ubuntu/Debian
apt-get install ffmpeg

# Docker
RUN apt-get update && apt-get install -y ffmpeg

Disable streaming TTS if FFmpeg unavailable:

const server = new LLMRTCServer({
  streamingTTS: false  // Uses non-streaming fallback
});

Check audio element - Client must connect ttsTrack to audio element:

client.on('ttsTrack', (stream) => {
  const audio = new Audio();
  audio.srcObject = stream;
  audio.play().catch(err => console.error('Playback failed:', err));
});

Tool Call Errors

Symptoms:

TOOL_ERROR returned to client
LLM response incomplete after tool call

Log excerpt:

Tool execution failed: get_weather
Error: Cannot read properties of undefined (reading 'temperature')
Arguments: {"city":"New York"}

Solutions:

Validate JSON Schema matches expected arguments:

defineTool({
  name: 'get_weather',
  description: 'Get current weather',
  parameters: {
    type: 'object',
    properties: {
      city: { type: 'string', description: 'City name' },
      units: { type: 'string', enum: ['celsius', 'fahrenheit'] }
    },
    required: ['city']
  }
}, async ({ city, units = 'celsius' }) => {
  // Handle optional parameters with defaults
});

Add error handling in tool implementation:

// In the handler function passed to defineTool:
async (args) => {
  try {
    const data = await fetchWeather(args.city);
    return { temperature: data.temp, condition: data.condition };
  } catch (error) {
    // Return error object instead of throwing
    return { error: 'Weather service unavailable' };
  }
}

Ensure serializable results - No functions, circular references:

// Bad: Contains non-serializable data
return { data: rawResponse, fetch: () => {} };

// Good: Plain object
return { temperature: 72, condition: 'sunny' };

Session Drops / Reconnection Issues

Symptoms:

Client repeatedly shows reconnecting then failed
SESSION_NOT_FOUND errors on reconnect

Log excerpt:

Reconnect attempt 1/5...
Session abc123 not found
Reconnect attempt 2/5...
Max retries exceeded, connection failed

Solutions:

Handle reconnection gracefully on client:

client.on('stateChange', (state) => {
  if (state === 'failed') {
    // Start fresh session instead of reconnecting
    client.start();  // Creates new session
  }
});

Check heartbeat timeout - Client should send pings:

// Server logs if no heartbeat received
Heartbeat timeout for session abc123

Rate Limiting

Symptoms:

RATE_LIMITED error code
Responses suddenly stop working

Log excerpt:

OpenAI API error: 429 Too Many Requests
Rate limit exceeded. Please retry after 60 seconds.

Solutions:

Implement retry logic - The PlaybookOrchestrator includes built-in retry logic with exponential backoff
Reduce request rate - Increase silence threshold, debounce inputs
Use tiered API plans from your provider

Debug Techniques

Browser DevTools

Network tab → WS to inspect WebSocket messages
Look for error message types with codes
Check for failed ICE candidates

Server Logging

Enable verbose hooks for debugging:

import { createVerboseHooks } from '@llmrtc/llmrtc-core';

const server = new LLMRTCServer({
  hooks: createVerboseHooks()
});

Or create targeted logging:

hooks: {
  onError: (error, context) => {
    console.error(`[${context}] Error:`, error);
  },
  onToolError: (ctx, request, error) => {
    console.error(`Tool ${request.name} failed:`, error);
    console.error('Arguments:', request.arguments);
  }
}

Connection State Debugging

client.on('stateChange', (state) => {
  console.log(`Connection state: ${state}`);
});

client.on('reconnecting', (attempt, max) => {
  console.log(`Reconnect attempt ${attempt}/${max}`);
});

Networking & TURN - ICE/TURN configuration
Observability & Hooks - Logging and metrics
Logging & Metrics - Production monitoring

Error Codes Reference​

Connection Errors​

Provider Errors​

Processing Errors​

Playbook/Tool Errors​

Generic Errors​

Common Issues​

No Audio / Microphone Blocked​

WebRTC Connection Fails​

High Latency​

TTS Produces Silence​

Tool Call Errors​

Session Drops / Reconnection Issues​

Rate Limiting​

Debug Techniques​

Browser DevTools​

Server Logging​

Connection State Debugging​

Related Documentation​

Error Codes Reference

Connection Errors

Provider Errors

Processing Errors

Playbook/Tool Errors

Generic Errors

Common Issues

No Audio / Microphone Blocked

WebRTC Connection Fails

High Latency

TTS Produces Silence

Tool Call Errors

Session Drops / Reconnection Issues

Rate Limiting

Debug Techniques

Browser DevTools

Server Logging

Connection State Debugging

Related Documentation