Skip to main content

Security

Security considerations for deploying LLMRTC in production environments.

API Key Management​

Storage​

Never hardcode API keys. Use environment variables or a secrets manager:

# Environment variables
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...
export ELEVENLABS_API_KEY=xi-...
// Load from environment
const server = new LLMRTCServer({
  providers: {
    llm: new OpenAILLMProvider({
      apiKey: process.env.OPENAI_API_KEY!  // Never hardcode
    })
  }
});

Secret Managers​

For production, use a secrets manager:

// AWS Secrets Manager
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';

async function getSecrets() {
  const client = new SecretsManagerClient({ region: 'us-east-1' });
  const response = await client.send(
    new GetSecretValueCommand({ SecretId: 'llmrtc/api-keys' })
  );
  return JSON.parse(response.SecretString!);
}

const secrets = await getSecrets();
const server = new LLMRTCServer({
  providers: {
    llm: new OpenAILLMProvider({ apiKey: secrets.OPENAI_API_KEY })
  }
});

Key Rotation​

When rotating keys:

  1. Deploy new key to secrets manager
  2. Restart server instances
  3. Revoke old key

Authentication​

LLMRTC doesn't include authentication—implement it in your application:

JWT Authentication​

import { LLMRTCServer } from '@llmrtc/llmrtc-backend';
import { verify } from 'jsonwebtoken';

const server = new LLMRTCServer({ providers });
const app = server.getApp();

// Middleware to verify JWT
app.use((req, res, next) => {
  // Skip health check
  if (req.path === '/health') {
    return next();
  }

  const authHeader = req.headers.authorization;
  if (!authHeader?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'Missing token' });
  }

  const token = authHeader.substring(7);
  try {
    const payload = verify(token, process.env.JWT_SECRET!);
    req.user = payload;
    next();
  } catch {
    return res.status(401).json({ error: 'Invalid token' });
  }
});

WebSocket Authentication​

WebSocket connections need special handling:

// Option 1: Query parameter token
// Client connects to: ws://server:8787?token=eyJ...

// Option 2: First message authentication
// Client sends auth message immediately after connect

// Option 3: Pre-authenticated ticket
app.post('/api/tickets', async (req, res) => {
  const user = req.user;
  const ticket = crypto.randomUUID();
  await redis.setex(`ticket:${ticket}`, 30, JSON.stringify(user));
  res.json({ ticket });
});

// Client connects with ticket, server validates on connect

Session Correlation​

Map LLMRTC sessions to your user accounts:

const userSessions = new Map<string, string>();  // sessionId -> userId

server.on('connection', ({ id }) => {
  // Look up pre-authenticated user from ticket or token
  const userId = getUserFromConnection(id);
  userSessions.set(id, userId);
});

server.on('disconnect', ({ id }) => {
  userSessions.delete(id);
});

Network Security​

TLS/HTTPS​

Always use TLS in production:

# nginx.conf
server {
    listen 443 ssl http2;
    server_name voice.example.com;

    ssl_certificate /etc/letsencrypt/live/voice.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/voice.example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:8787;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_read_timeout 86400;
    }
}

CORS Configuration​

Restrict allowed origins:

const server = new LLMRTCServer({
  providers,
  cors: {
    origin: ['https://app.example.com', 'https://admin.example.com'],
    credentials: true,
    methods: ['GET', 'POST', 'OPTIONS'],
    allowedHeaders: ['Content-Type', 'Authorization']
  }
});

Firewall Rules​

Only expose necessary ports:

# Allow HTTPS and WSS
ufw allow 443/tcp

# Allow WebRTC media (if not using TURN)
ufw allow 10000:20000/udp

# Block direct access to backend port
ufw deny 8787/tcp

Rate Limiting​

Prevent abuse with rate limiting:

At the Gateway​

# nginx.conf
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;

server {
    location / {
        limit_req zone=api burst=20 nodelay;
        proxy_pass http://127.0.0.1:8787;
    }
}

Per-User Limits​

import { RateLimiter } from 'limiter';

const userLimiters = new Map<string, RateLimiter>();

app.use(async (req, res, next) => {
  const userId = req.user?.id;
  if (!userId) return next();

  let limiter = userLimiters.get(userId);
  if (!limiter) {
    limiter = new RateLimiter({
      tokensPerInterval: 60,
      interval: 'minute'
    });
    userLimiters.set(userId, limiter);
  }

  if (await limiter.removeTokens(1) < 0) {
    return res.status(429).json({ error: 'Rate limit exceeded' });
  }

  next();
});

Concurrent Session Limits​

const activeSessions = new Map<string, Set<string>>();  // userId -> sessionIds
const MAX_SESSIONS_PER_USER = 3;

server.on('connection', ({ id }) => {
  const userId = getUserId(id);
  const sessions = activeSessions.get(userId) || new Set();

  if (sessions.size >= MAX_SESSIONS_PER_USER) {
    // Disconnect oldest or reject new connection
    const oldest = sessions.values().next().value;
    disconnectSession(oldest);
    sessions.delete(oldest);
  }

  sessions.add(id);
  activeSessions.set(userId, sessions);
});

Data Protection​

PII in Transcripts​

Transcripts may contain personally identifiable information:

// Pass hooks via constructor
const server = new LLMRTCServer({
  providers,
  hooks: {
    onSTTEnd: async (ctx, result) => {
      // Log transcript without PII
      const sanitized = redactPII(result.text);
      logger.info({ sessionId: ctx.sessionId, transcript: sanitized });

      // Store securely if needed
      if (shouldStore()) {
        await encryptAndStore(ctx.sessionId, result.text);
      }
    }
  }
});

Data Retention​

Define retention policies:

// Automatic cleanup
setInterval(async () => {
  const cutoff = Date.now() - 30 * 24 * 60 * 60 * 1000;  // 30 days
  await db.query('DELETE FROM transcripts WHERE created_at < ?', [cutoff]);
}, 24 * 60 * 60 * 1000);  // Daily

Encryption at Rest​

Encrypt sensitive data:

import { createCipheriv, createDecipheriv, randomBytes } from 'crypto';

function encrypt(text: string, key: Buffer): string {
  const iv = randomBytes(16);
  const cipher = createCipheriv('aes-256-gcm', key, iv);
  const encrypted = Buffer.concat([cipher.update(text), cipher.final()]);
  const tag = cipher.getAuthTag();
  return Buffer.concat([iv, tag, encrypted]).toString('base64');
}

Input Validation​

Tool Arguments​

Validate tool arguments before execution:

import { z } from 'zod';

const BookingSchema = z.object({
  date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
  time: z.string().regex(/^\d{2}:\d{2}$/),
  partySize: z.number().int().min(1).max(20)
});

const bookingTool = defineTool({
  name: 'book_table',
  description: 'Book a table at a restaurant',
  parameters: {
    type: 'object',
    properties: {
      date: { type: 'string', description: 'Booking date (YYYY-MM-DD)' },
      time: { type: 'string', description: 'Booking time (HH:MM)' },
      partySize: { type: 'number', description: 'Number of guests' }
    },
    required: ['date', 'time', 'partySize']
  }
}, async (args) => {
  const validated = BookingSchema.parse(args);
  return await bookTable(validated);
});

Prompt Injection​

Guard against prompt injection:

function sanitizeUserInput(text: string): string {
  // Remove potential injection patterns
  return text
    .replace(/\[SYSTEM\]/gi, '')
    .replace(/ignore previous instructions/gi, '')
    .slice(0, 1000);  // Length limit
}

Audit Logging​

Log security-relevant events:

const auditLog = {
  logConnection(sessionId: string, userId: string, ip: string) {
    logger.info({
      event: 'connection',
      sessionId,
      userId,
      ip,
      timestamp: new Date().toISOString()
    });
  },

  logToolExecution(sessionId: string, tool: string, args: unknown) {
    logger.info({
      event: 'tool_execution',
      sessionId,
      tool,
      argsHash: hash(JSON.stringify(args)),
      timestamp: new Date().toISOString()
    });
  },

  logError(sessionId: string, error: Error) {
    logger.error({
      event: 'error',
      sessionId,
      error: error.message,
      stack: error.stack,
      timestamp: new Date().toISOString()
    });
  }
};