gemini-api-dev

name	gemini-api-dev
description	Build applications using Google Gemini API. Handle chat completions, multimodal inputs, function calling, streaming, and grounding with Google Search. Use when building applications using google gemini api. handle chat completions, multimodal.
domain	core
author	oyi77
license	Apache-2.0
subdomain	core-platform
tags	["ai","gemini","google","llm","multimodal","api"]
version	1.0.0

Gemini Api Dev

When to Use

Trigger phrases:

• "gemini api dev"

• "When building apps powered by Google Gemini models"

• "When processing multimodal inputs (text + images + audio + video)"

• "When implementing function calling or tool use with Gemini"

• When building apps powered by Google Gemini models

• When processing multimodal inputs (text + images + audio + video)

• When implementing function calling or tool use with Gemini

• When using Gemini for grounding with Google Search

When NOT to Use

• For OpenAI-compatible APIs (use OpenAI skills)
• For local model inference (use Ollama skills)
• For fine-tuning (use Vertex AI training skills)

Overview

The Google Gemini API provides access to Google's most capable AI models — Gemini 2.5 Pro, 2.5 Flash, and Gemini Nano — through a unified SDK. It supports text generation, multimodal understanding (images, audio, video, PDFs), function calling, and grounding with Google Search. The API is available via the google-genai Python SDK and @google/genai TypeScript SDK, both providing first-class async support, streaming generators, and type-safe response handling.

Gemini 2.5 Pro excels at complex reasoning, code generation, and multi-turn conversations with a 1-million-token context window. Gemini 2.5 Flash is optimized for speed and cost-efficiency while maintaining strong reasoning capabilities, making it suitable for high-volume production workloads. Nano models run on-device for Android and Chrome, handling summarization, smart reply, and text classification without network calls.

The API architecture follows a content-based message model: each request sends contents arrays with role (user or model) and parts — which can be text, inline_data (base64-encoded images/audio), file_data (PDFs, videos), function_call, or function_response. Configuration options include systemInstruction for persona setting, temperature (0.0–2.0), topP, topK, maxOutputTokens, stopSequences, and safetySettings for content filtering.

Grounding with Google Search enables responses that reference real-time information, making Gemini suitable for news analysis, fact-checking, and research tasks. Enabled via googleSearch grounding in the generation config, it returns groundingMetadata with source URLs, supporting citations alongside generated text. This feature is especially valuable for applications requiring current, verifiable information.

Streaming is a first-class pattern across both SDKs — generateContentStream returns async generators that yield chunks as they become available, enabling responsive UI updates, early processing of partial results, and progressive rendering of long outputs without buffering.

Workflow

1. Set up SDK — Install google-genai (Python) or @google/genai (TypeScript). Configure API key from Google AI Studio or service account for Vertex AI. Load from GEMINI_API_KEY environment variable.
2. Configure auth — API key for Google AI Studio, service account for Vertex AI. For production, use service account with workload identity federation to avoid hardcoded keys.
3. Define system instruction — Set model persona, output format constraints, tone, and domain expertise via systemInstruction in generation config. System instructions are treated as developer messages and significantly improve output consistency.
4. Prepare contents — Build message history with alternating user/model roles. Include multimodal parts: inline_data for base64 images/audio, file_data for PDFs/videos uploaded via File API, or text blocks. Use role: "user" for input and role: "model" for prior assistant turns.
5. Configure generation parameters — Set temperature (0.0–2.0), top_p, top_k, max_output_tokens, stop_sequences, and safety_settings per request. For deterministic output, set a seed parameter (available in Pro models).
6. Handle responses — Stream via generateContentStream async generator for real-time output. Handle function_call parts by executing tools and resuming with function_response parts. Process grounded responses that include groundingMetadata with source citations and groundingChunks.
7. Error handling — Catch ClientError (4xx) and ServerError (5xx). Implement exponential backoff with jitter for 429 RESOURCE_EXHAUSTED rate limits. Fallback between Pro and Flash models on quota exhaustion per region.

Anti-Rationalization Table

Rationalization	Reality
"I will just use curl"	SDK handles retries, streaming, auth, and type safety
"Gemini is just for text"	Gemini 2.5 processes images, audio, video, and PDFs natively
"One prompt fits all"	System instructions and grounding dramatically improve accuracy
"Temperature 0 is deterministic"	Even at 0, sampling can produce different outputs; use seed parameter for reproducibility with Pro models
"I don't need streaming for short responses"	Streaming is the only way to get real-time function call progress and partial safety assessments
"Safety settings can stay default"	Default blocks are conservative — tune per use case or responses may be silently blocked

Code Example (TypeScript)

import { GoogleGenAI } from '@google/genai';

const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const response = await ai.models.generateContentStream({
  model: 'gemini-2.5-pro',
  contents: [{ role: 'user', parts: [{ text: 'Analyze this chart' }] }],
  config: {
    systemInstruction: 'You are a data analyst. Be precise and cite numbers.',
    temperature: 0.3,
  }
});

for await (const chunk of response) {
  process.stdout.write(chunk.text);
}

Code Example (Python)

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_API_KEY")

# Basic text generation
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Explain how RAG works in two sentences.",
)
print(response.text)

# Multimodal input with image
import httpx
img = httpx.get("https://example.com/chart.png").content
response = client.models.generate_content(
    model="gemini-2.5-pro",
    contents=[
        "Analyze this chart and summarize the trend.",
        types.Part.from_bytes(data=img, mime_type="image/png"),
    ],
)
print(response.text)

# Streaming
stream = client.models.generate_content_stream(
    model="gemini-2.5-pro",
    contents="Write a short story about AI.",
    config=types.GenerateContentConfig(
        temperature=0.7,
        max_output_tokens=2048,
    ),
)
for chunk in stream:
    print(chunk.text, end="")

# Function calling
def get_weather(city: str) -> str:
    return f"Weather in {city}: sunny, 25\u00b0C"

tools = [
    types.Tool(function_declarations=[
        types.FunctionDeclaration(
            name="get_weather",
            description="Get current weather for a city",
            parameters={
                "type": "object",
                "properties": {
                    : {: , : }
                },
                : [],
            },
        ),
    ]),
]

response = client.models.generate_content(
    model=,
    contents=,
    config=types.GenerateContentConfig(tools=tools),
)

 response.candidates[].content.parts[].function_call:
    fc = response.candidates[].content.parts[].function_call
    result = get_weather(fc.args[])
    
    response = client.models.generate_content(
        model=,
        contents=[
            ,
            types.Part.from_function_call(fc),
            types.Part.from_function_response(
                name=fc.name,
                response={: result},
            ),
        ],
        config=types.GenerateContentConfig(tools=tools),
    )
    (response.text)

Setup / Configuration

Installation

Python:

pip install google-genai

TypeScript / Node.js:

npm install @google/genai

Authentication

1. Go to Google AI Studio and click Get API Key
2. Create an API key (free tier includes 60 requests per minute)
3. Set it as environment variable:

   export GEMINI_API_KEY="your-api-key-here"
   

For production deployments, use a Vertex AI service account with workload identity federation to avoid embedding API keys in application code.

Safety Settings

Gemini safety filters operate on four categories: harassment, hate speech, sexually explicit content, and dangerous content. Each has adjustable thresholds:

Threshold	Behavior
BLOCK_NONE	Never block
BLOCK_ONLY_HIGH	Block only high-confidence harmful content
BLOCK_MEDIUM_AND_ABOVE (default)	Block medium and high content
BLOCK_LOW_AND_ABOVE	Most restrictive — blocks everything

Override via generation config:

config = types.GenerateContentConfig(
    safety_settings=[
        types.SafetySetting(
            category="HARM_CATEGORY_HARASSMENT",
            threshold="BLOCK_ONLY_HIGH",
        ),
    ],
)

Model Selection Guide

Model	Use Case	Context	Speed
gemini-2.5-pro	Complex reasoning, code, multimodal	1M tokens	Slower
gemini-2.5-flash	Chat, fast responses, high volume	1M tokens	Fast
gemini-2.5-flash-8b	Lightweight, specialized tasks	1M tokens	Fastest

Common Issues / Troubleshooting

Error / Symptom	Root Cause	Solution
403 PERMISSION_DENIED	Invalid or expired API key	Regenerate key in AI Studio, verify GEMINI_API_KEY env var
429 RESOURCE_EXHAUSTED	Rate limit exceeded (60 RPM free tier)	Implement exponential backoff, switch to Flash model, upgrade to paid tier
400 INVALID_ARGUMENT — blocked content	Safety filter triggered	Review safety settings, adjust thresholds, check input content
Empty or truncated responses	max_output_tokens too low	Increase to 8192 for complex tasks
Function call not triggered	Model chose not to call — vague prompt	Add explicit instruction: "Use the available functions"
Streaming stalls / hangs	Network issues or server timeout	Set client timeout, implement retry with generateContent fallback
400 — invalid multimodal format	Image encoded incorrectly	Use valid base64; supported MIME: PNG, JPEG, WEBP, HEIC, HEIF
High latency on simple tasks	Using Pro model unnecessarily	Switch to Flash for speed, reserve Pro for reasoning-heavy work

Monetization

Approach	Timeframe	Description
Gemini-powered SaaS API	2-4 weeks	Build vertical API for content analysis, document processing, or image understanding; charge per-request or subscription
Chrome extension with Nano	1-2 weeks	On-device AI extension for summarization/rewriting — zero server costs using Gemini Nano
Multimodal chatbot service	2-3 weeks	Custom chatbots for businesses that analyze images, PDFs, and documents using Gemini Pro vision
AI content moderation API	3-4 weeks	Wrap Gemini safety classification into a moderation SaaS for user-generated content platforms
Code review automation tool	2-3 weeks	PR review assistant using Gemini 2.5 Pro code analysis — integrate with GitHub Apps marketplace
Educational content generator	1-2 weeks	Generate quiz questions and study guides from lecture materials (PDFs + video transcripts) using multimodal input

Process

1. Design — Define interface, identify patterns, plan implementation
2. Implement — Write code following existing conventions, add tests
3. Verify — Run tests, check integration, validate behavior

Verification

• API key authenticated successfully via Google AI Studio
• Chat completions return valid responses for multiple models (Pro, Flash)
• Streaming works without buffering issues or missing chunks
• Function calling triggers correctly and resumes conversation
• Safety settings block harmful content at configured thresholds
• Multimodal input (image/audio/PDF) processed without errors
• Grounding with Google Search returns citations in response
• Error handling catches rate limit (429) and permission (403) errors
• System instruction correctly influences model behavior
• Token usage tracked and within quota limits