ai-gateway

name

ai-gateway

description

Use when making AI API calls through the personal AI Gateway at llm.kashifkhan.dev. Triggers include "use the gateway", "call gemini", "use big-pickle", "use minimax", "make an AI API call", "chat completion", "ask the AI to...", "use llm.kashifkhan.dev", "which model should I use", or any request to implement AI chat/completion functionality in code. Provides model selection, request construction, streaming patterns, and language-agnostic implementation guidance.

AI Gateway Skill

Personal AI Gateway that proxies multiple AI models through a single OpenAI-compatible API.

Base URL: https://llm.kashifkhan.dev
Auth: Authorization: Bearer <YOUR_API_KEY> on all endpoints except /health
Protocol: OpenAI-compatible — works with any OpenAI SDK by setting base_url

---

Activation Triggers

Load this skill when the user:

• Says "use the gateway", "call the AI gateway", "use llm.kashifkhan.dev"
• Mentions a specific model: gemini, big-pickle, minimax, kimi, gpt-5-nano, glm
• Asks "which model should I use" or "what models are available"
• Wants to implement AI chat/completion in any language
• Asks to "make an AI API call", "build a chatbot", "add AI to this app"
• Needs streaming AI responses in their code
• Asks about system prompts, multi-turn conversations, or message history

---

Model Selection

Decision Tree

Need a free model?
├── Yes → Use streaming (stream: true), pick:
│   ├── General purpose          → big-pickle      (alias: bp)
│   ├── Multilingual / Pashto    → minimax-m2.5-free (alias: minimax, mm)
│   ├── Lightweight / fast       → gpt-5-nano      (alias: nano)
│   ├── GLM architecture         → glm-5-free      (alias: glm)
│   └── Kimi model               → kimi-k2.5-free  (alias: kimi)
└── No (premium, Gemini sub) → pick:
    ├── Default / best all-around → gemini-2.5-flash (alias: gemini)
    ├── Most capable / complex    → gemini-2.5-pro   (alias: gemini-pro) ← RECOMMENDED DEFAULT
    ├── Image understanding       → gemini-2.5-flash-image (alias: gemini-img) — USE STREAMING
    ├── Fast experimental         → gemini-3-flash-preview (alias: g3f)
    └── Most powerful available   → gemini-3-pro-preview   (alias: g3p)

Quick Reference Table

Model ID	Aliases	Free	Notes
big-pickle	bp, pickle, big-pickle-free	Yes	General purpose
glm-5-free	glm, glm5	Yes	GLM architecture
minimax-m2.5-free	minimax, mm	Yes	Best for Pashto/multilingual
gpt-5-nano	nano, gpt5	Yes	Lightweight GPT-5 variant
kimi-k2.5-free	kimi, k2.5	Yes	Kimi model
gemini-2.5-flash	gemini, gemini-flash, g25f	No	Default recommendation
gemini-2.5-pro	gemini-pro, g25p	No	Best for complex tasks
gemini-2.5-flash-image	gemini-img, g25fi	No	Image input — use streaming
gemini-3-flash-preview	g3f	No	Fast, experimental
gemini-3-pro-preview	g3p	No	Most powerful

Rule: Free models → always use "stream": true. Premium models → either mode works.

---

Endpoints

Health Check

GET /health

No auth required. Returns gateway status, version, backend connectivity, uptime.

List Models

GET /v1/models
Authorization: Bearer <YOUR_API_KEY>

List Backends

GET /v1/backends
Authorization: Bearer <YOUR_API_KEY>

Chat Completion

POST /v1/chat/completions
Authorization: Bearer <YOUR_API_KEY>
Content-Type: application/json

---

Request Schema

{
  "model": "gemini",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "Hello!" }
  ],
  "stream": false
}

Parameters

Field	Type	Required	Notes
model	string	Yes	Model ID or alias (e.g. gemini, g3p, minimax)
messages	array	Yes	Array of {role, content} objects
stream	boolean	No	Default: false. Use true for free models
backend	string	No	Force specific backend (e.g. opencode). Rarely needed.
temperature	float	No	Accepted but ignored — not forwarded to model
max_tokens	integer	No	Accepted but ignored — not forwarded to model
top_p	float	No	Accepted but ignored — not forwarded to model

Message Roles

Role	When to Use
system	Sets AI identity/behavior. Optional. Passed directly to the model.
user	User message
assistant	Prior AI response (for multi-turn context)

---

Response Formats

Non-Streaming Response

{
  "id": "chatcmpl-<session_id>",
  "object": "chat.completion",
  "created": 1771468405,
  "model": "gemini-2.5-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! How can I help you today?"
      },
      "finish_reason": "stop"
    }
  ]
}

Extract content: response.choices[0].message.content

Streaming Response (SSE)

Stream arrives as Server-Sent Events. Each data: line is a JSON chunk.

Chunk sequence:

1. Role chunk — empty content, announces "role": "assistant"
2. Content chunk(s) — the actual text (free models: word-by-word; Gemini Pro: larger batches)
3. Finish chunk — "finish_reason": "stop", empty delta
4. data: [DONE] — plain string (not JSON), signals end of stream

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{"role":"assistant"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":...,"model":"gemini-2.5-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Parsing rules:

• Skip lines not starting with data:
• Skip data: [DONE] (not JSON)
• Skip chunks where delta.content is absent or empty
• Concatenate all delta.content values for the full response

---

Implementation Patterns (Language-Agnostic)

When implementing in any language, the pattern is always:

1. Build JSON body:    {model, messages, stream?}
2. Set headers:        Authorization: Bearer <key>, Content-Type: application/json
3. POST to:            https://llm.kashifkhan.dev/v1/chat/completions
4a. Non-stream:        Parse JSON → choices[0].message.content
4b. Stream:            Read SSE lines → filter data: → parse JSON → delta.content → concatenate

HTTP Request Structure (Any Language)

Method:   POST
URL:      https://llm.kashifkhan.dev/v1/chat/completions
Headers:
  Authorization: Bearer <YOUR_API_KEY>
  Content-Type: application/json
Body:     JSON encoded request object

---

cURL Examples

Non-Streaming

curl -X POST https://llm.kashifkhan.dev/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Streaming (free model)

curl -X POST https://llm.kashifkhan.dev/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  --no-buffer \
  -d '{
    "model": "big-pickle",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

With System Prompt

curl -X POST https://llm.kashifkhan.dev/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant named Alex."},
      {"role": "user", "content": "Who are you?"}
    ]
  }'

Multi-Turn Conversation

curl -X POST https://llm.kashifkhan.dev/v1/chat/completions \
  -H "Authorization: Bearer <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini",
    "messages": [
      {"role": "system",    "content": "You are a helpful assistant."},
      {"role": "user",      "content": "My name is Zarqan."},
      {"role": "assistant", "content": "Nice to meet you, Zarqan!"},
      {"role": "user",      "content": "What is my name?"}
    ]
  }'

---

Error Reference

HTTP	Code	Meaning	Fix
400	invalid_model	Model ID not recognized	Check model ID or alias — no provider prefix (use gemini-2.5-flash not google/gemini-2.5-flash)
400	invalid_messages	Malformed messages array	Ensure each message has role and content string fields
401	missing_api_key	No Authorization header	Add Authorization: Bearer <YOUR_API_KEY>
401	invalid_api_key	Wrong API key	Verify key value
500	backend_unavailable	Model/provider returned error	Retry or switch model
503	backend_unavailable	OpenCode backend is down	Check gateway health at GET /health

Error Response Shape

{
  "error": {
    "message": "Human-readable description",
    "type": "authentication_error | invalid_request_error | backend_error",
    "code": "missing_api_key | invalid_api_key | invalid_model | invalid_messages | backend_unavailable",
    "status": 400
  }
}

---

Gotchas & Important Notes

1. temperature, max_tokens, top_p are silently ignored. The gateway accepts them (for SDK compatibility) but never forwards them to the model. Do not rely on these for output control.

2. Free models require streaming. Non-streaming works but is less reliable. Always use "stream": true with big-pickle, glm-5-free, minimax-m2.5-free, gpt-5-nano, kimi-k2.5-free.

3. gemini-2.5-flash-image should use streaming. Non-streaming may return unexpected content.

4. No provider prefix in model IDs. Use gemini-2.5-flash, NOT google/gemini-2.5-flash.

5. data: [DONE] is not JSON. When parsing SSE, check for this literal string before calling JSON.parse() / json.loads().

6. Role chunk has no content. The first SSE chunk announces "role": "assistant" with an empty delta — skip it when accumulating content.

7. Rate limiting is configured (60 req/min per key) but not yet enforced. The middleware is a no-op placeholder — do not depend on this for production throttling logic.

8. OpenAI SDK compatible. Any language's OpenAI client works by setting base_url / baseURL to https://llm.kashifkhan.dev/v1.

---

Quick Reference Card

Base URL:   https://llm.kashifkhan.dev
Auth:       Authorization: Bearer <YOUR_API_KEY>
Endpoint:   POST /v1/chat/completions
Default:    model = "gemini"  (gemini-2.5-flash)
Free:       big-pickle, glm, minimax, nano, kimi  → always stream: true
Premium:    gemini, gemini-pro, g3p, g3f, gemini-img → stream optional
Ignored:    temperature, max_tokens, top_p
SSE end:    data: [DONE]   (literal string, not JSON)