gemini-proxy

name

gemini-proxy

description

Use when making AI API calls through the gemini-proxy at llm.kashifkhan.dev. This is a self-hosted OpenAI-compatible proxy for Google Gemini models. Triggers include "use the gemini proxy", "call llm.kashifkhan.dev", "use gemini", "make a Gemini API call", "chat completion with Gemini", or any request to implement AI functionality using this proxy. Provides model selection, authentication, request construction, streaming patterns, tool calling, and error handling for the gemini-proxy API.

Gemini Proxy

Self-hosted OpenAI-compatible API for Google Gemini, available at https://llm.kashifkhan.dev.

Base URL: https://llm.kashifkhan.dev/v1
Auth: Authorization: Bearer <API_KEY> on all /v1/* endpoints
Protocol: OpenAI-compatible — works with any OpenAI SDK by setting baseURL / base_url

---

Quick Start

TypeScript / JavaScript

import OpenAI from "openai";
const ai = new OpenAI({
  apiKey: process.env.AI_API_KEY,
  baseURL: "https://llm.kashifkhan.dev/v1",
});
const response = await ai.chat.completions.create({
  model: "gemini-2.0-flash",
  messages: [{ role: "user", content: "Hello!" }],
});
console.log(response.choices[0].message.content);

Python

from openai import OpenAI
ai = OpenAI(api_key=os.environ["AI_API_KEY"], base_url="https://llm.kashifkhan.dev/v1")
response = ai.chat.completions.create(
    model="gemini-2.0-flash",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

---

Models

Model ID	Best For
gemini-2.5-pro	Complex reasoning, strict instruction following
gemini-2.5-flash	Fast + capable, good default for most tasks
gemini-2.0-flash	Recommended default — balanced speed/quality
gemini-2.0-flash-lite	Fastest, lowest latency, simple tasks
gemini-1.5-pro	Large context windows
gemini-1.5-flash	Previous generation, fast

Default: gemini-2.0-flash

---

Endpoints

Method	Path	Auth	Description
GET	/health	None	Health check → {"status":"ok"}
GET	/auth/status	None	Check proxy's Google session is active
GET	/v1/models	API key	List available models
POST	/v1/chat/completions	API key	Chat completions (streaming + non-)

Auth Status

curl https://llm.kashifkhan.dev/auth/status
# {"authenticated": true, "email": "...", "projectId": "...", "expiresAt": "..."}

If authenticated is false, the server needs re-authentication with Google — contact Kashif.

---

Request Parameters

Field	Type	Required	Notes
model	string	Yes	Model ID from table above
messages	array	Yes	Conversation history
stream	boolean	No	true for SSE streaming. Default: false
temperature	number	No	Range: 0.0–2.0
max_tokens	number	No	Max tokens in response
top_p	number	No	Nucleus sampling. Range: 0.0–1.0
stop	string or string[]	No	Stop sequences
tools	array	No	Function definitions for tool calling
tool_choice	string or object	No	Accepted for compatibility but has no effect on Gemini

Message Roles

"system", "user", "assistant", "tool"

Image Input

Only data: URIs (base64). Remote URLs not supported.

{
  "role": "user",
  "content": [
    { "type": "text", "text": "What is in this image?" },
    { "type": "image_url", "image_url": { "url": "data:image/png;base64,<base64>" } }
  ]
}

---

System Prompts

Pass as first message with "role": "system". Only the first system message takes effect. Repeat it on every request for multi-turn conversations.

gemini-2.5-pro follows system instructions more reliably than gemini-2.0-flash.

---

Streaming

Set "stream": true to get Server-Sent Events.

data: {"choices":[{"delta":{"role":"assistant"}}]}
data: {"choices":[{"delta":{"content":"Hello"}}]}
data: {"choices":[{"delta":{"content":" there!"},"finish_reason":"stop"}]}
data: [DONE]

When to stream:

• Chat UIs / real-time display → stream: true (first token ~1.7s)
• Long responses (>500 tokens) → stream: true
• Background batch processing → stream: false

const stream = await ai.chat.completions.create({
  model: "gemini-2.0-flash",
  messages: [{ role: "user", content: "Write a story." }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

---

Tool Calling

const response = await ai.chat.completions.create({
  model: "gemini-2.0-flash",
  messages: [{ role: "user", content: "What is the weather in Karachi?" }],
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get current weather for a city",
      parameters: {
        type: "object",
        properties: { city: { type: "string" } },
        required: ["city"],
      },
    },
  }],
});
// Model returns: { tool_calls: [{ id, function: { name, arguments } }] }
// Then add tool result with role: "tool", tool_call_id, content

---

Error Reference

HTTP	code	Meaning	Action
400	null	Bad request / missing field	Fix request body
401	invalid_api_key	Missing or wrong API key	Check Authorization header
401	not_authenticated	Proxy's Google session expired	Contact Kashif to re-auth server
401	no_project	GCP project not resolved	Contact Kashif to re-auth server
500	null	Internal server error	Retry; contact Kashif if it persists
502	null	Cannot reach Google upstream	Retry after a short wait

---

Gotchas

1. Only first system message takes effect — additional system messages are ignored
2. tool_choice has no effect — Gemini does not support per-request tool choice
3. Only data: image URIs — remote HTTP/HTTPS image URLs are not supported
4. /health and /auth/status need no auth — all /v1/* endpoints require API key