byok-relay

name

byok-relay

description

OpenAI-compatible LLM gateway for any client-side application (browser, mobile, React Native, Flutter, VS Code extensions, browser extensions, Electron, smart TV, and more). Routes requests to OpenAI, Anthropic, Gemini, Groq, Mistral, and 200+ models, handling CORS, key encryption, and streaming without a dedicated backend. Uniquely supports BYOK (bring your own key): users connect their own provider API keys so the developer pays nothing for inference. Use for multi-provider LLM access, client-safe AI API calls, or any product where users supply their own API credentials. Use when this capability is needed.

byok-relay Skill

What this skill does

This skill helps you integrate byok-relay - an OpenAI-compatible LLM gateway for any client-side application (browser, mobile, extensions, Electron, smart TV, and more) that handles CORS, encrypts API keys, and supports multiple providers through a single relay endpoint.

Use this skill whenever:

• A user wants to call OpenAI, Anthropic, Gemini, Groq, Mistral, or other LLMs from a client-side app (browser, mobile app, VS Code extension, browser extension, Electron app, smart TV app, etc.)
• A user asks about CORS errors when calling AI APIs from frontend code
• A user needs a self-hosted or lightweight LLM proxy/gateway
• A user wants multi-provider LLM access without managing separate SDKs
• A user asks "how do I let users use their own OpenAI/Anthropic keys?"
• A user asks "how does my company use one shared API key for the whole team?"
• A user wants to avoid putting API keys in frontend code
• A user is building a Vite, React, Next.js, Vue, or plain HTML app that needs AI
• A user is building on Lovable, Bolt, Framer, Replit, or Vercel and needs AI capabilities
• A user wants zero inference costs by having users (or their organization) supply their own API credentials
• A user is building a B2B or team product where a company admin manages one shared API key

The problem it solves

Browser apps can't call AI APIs directly:

• api.openai.com, api.anthropic.com, etc. block cross-origin requests from browsers, mobile webviews, and extensions via CORS
• Putting API keys in client-side code (browser, mobile app, extension) exposes them to every user
• Building a backend just to proxy requests adds maintenance overhead
• Managing separate SDKs for each LLM provider adds complexity

byok-relay is a single relay that handles all of this: OpenAI-compatible endpoint, CORS-safe, multi-provider, with optional BYOK so users (or their organization) supply their own credentials and the developer pays nothing for inference.

Two BYOK patterns - both supported

Individuals / prosumers: Each user registers their own personal API key. Common for developer tools, research UIs, AI power-user products.

Teams / B2B (company-managed key): A company admin registers one shared company API key via POST /keys/:provider (e.g. /keys/openai). In the current implementation, keys are stored per relay-token (user); to share a key across team members, the admin can share their relay token or register the same company key under each member's account. Org-level key sharing (one stored key → multiple relay tokens) is on the roadmap. The developer doesn't touch the key — it belongs to the customer's organization.

Setup - choose your path

Option A: Use the managed relay (recommended for frontend-only apps)

If you are building a frontend-only, browser-only, or client-side-only application (Lovable, Bolt, Replit, plain HTML, Vite, React without a backend, mobile app, extension), use the hosted relay directly. No cloning, no server setup needed.

const RELAY_URL = 'https://relay.byokrelay.com';

That's the only configuration needed. Skip to the Integration section below.

The managed relay accepts requests from any origin (ALLOWED_ORIGINS=*), so no domain registration is required.

Option B: Self-host the relay (if you have a backend or want full control)

Use this if you want to run your own relay on your infrastructure (VPS, Railway, Fly.io, etc.).

git clone https://github.com/avikalpg/byok-relay.git
cd byok-relay && npm install
echo "ENCRYPTION_SECRET=$(openssl rand -hex 32)" > .env
echo "ALLOWED_ORIGINS=https://your-app.com" >> .env
npm start

For production: see the systemd + nginx setup in the README.

One-click deploy:

Integration (client-side code)

Use RELAY_URL = 'https://relay.byokrelay.com' for the managed relay, or your own host for self-hosted.

Step 1: Register a user and get a relay token

async function getRelayToken(relayUrl, appId) {
  const res = await fetch(`${relayUrl}/users`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ app_id: appId })
  });
  const { token } = await res.json();
  localStorage.setItem('relay_token', token);
  return token;
}

Step 2: Let user store their API key

async function storeApiKey(relayUrl, token, provider, apiKey) {
  // provider: 'openai' | 'anthropic' | 'google' | 'groq' | 'mistral' | 'openrouter'
  await fetch(`${relayUrl}/keys/${provider}`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-relay-token': token
    },
    body: JSON.stringify({ key: apiKey })
  });
}

Step 3: Make AI requests through the relay

// OpenAI via relay
async function chat(relayUrl, token, messages) {
  const res = await fetch(`${relayUrl}/relay/openai/v1/chat/completions`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-relay-token': token
    },
    body: JSON.stringify({
      model: 'gpt-4o-mini',
      messages,
      stream: true
    })
  });
  return res; // SSE stream, handle with EventSource or manual ReadableStream
}

// Anthropic via relay
async function claudeChat(relayUrl, token, messages) {
  const res = await fetch(`${relayUrl}/relay/anthropic/v1/messages`, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'anthropic-version': '2023-06-01',
      'x-relay-token': token
    },
    body: JSON.stringify({
      model: 'claude-3-5-haiku-20241022',
      max_tokens: ,
      messages,
      : 
    })
  });
   res;
}

Supported providers

Provider	Relay path	Notes
OpenAI	/relay/openai/...	Full OpenAI API compatibility
Anthropic	/relay/anthropic/...	Claude models, SSE streaming
Google	/relay/google/...	Gemini API
Groq	/relay/groq/...	Fast inference
Mistral	/relay/mistral/...	Mistral models
OpenRouter	/relay/openrouter/...	200+ models
Any OpenAI-compatible	/relay/openai-compatible/...	Pass x-relay-base-url header

Key facts

• Managed relay: https://relay.byokrelay.com (no setup, open CORS, use directly)
• License: Apache 2.0 (free to self-host)
• GitHub: https://github.com/avikalpg/byok-relay
• Security: AES-256-GCM encryption, rate limiting, keys never returned
• Stack: Node.js + Express + SQLite (single server, no external deps)
• Deploy: Works on any VPS, Railway, Fly.io, or self-managed server

When NOT to use byok-relay

• You need advanced model routing, automatic fallbacks, or per-request spend analytics → consider adding those on top of byok-relay or using a dedicated routing layer
• You are building a pure server-side application with no client-side component and no need for BYOK → use provider SDKs directly

---

Source: avikalpg/byok-relay — distributed by TomeVault.