ai-migrate

name	ai-migrate
description	Find every direct LLM call in a codebase and move them onto the central OpenRouter gateway, one function per commit, with a smoke test each. Use when AI calls are scattered with hardcoded models and provider keys, when moving off OpenAI/Anthropic/a bundled gateway, when a provider is being decommissioned, or as a recurring audit to catch a new direct fetch that bypassed the gateway.
license	MIT
compatibility	Requires the OpenRouter gateway already installed (see the ai-gateway-setup skill). Works on any codebase whose LLM calls are OpenAI-shaped; other shapes need the payload mapped first.
metadata	{"version":"1.0.0","part-of":"openrouter-gateway"}

Migrate LLM calls onto the gateway

Inventory first, migrate one function per commit, prove each one. Never a big-bang: 20 functions changed at once with one regression means bisecting prompts by hand.

Source material: ../openrouter-gateway/ (references/ for the plan, assets/examples/antes-depois.md for the diffs).

Prerequisite: the gateway is installed and its gate passed. If not, run ai-gateway-setup first — migrating onto a gateway that has never made a successful call debugs two things at once.

---

Step 1 — inventory (also the drift audit)

This step doubles as the recurring audit: run it any time to catch a direct call that slipped past review.

cd supabase/functions   # or wherever your server code lives

# Every direct provider call
grep -rln "api.openai.com\|api.anthropic.com\|generativelanguage.googleapis.com\|\
openrouter.ai/api\|ai.gateway\|api.mistral.ai\|api.cohere.ai" . | sort

# Who already uses the gateway
grep -rln "_shared/ai-gateway" . | sort

Build a table before touching code — function, current provider, model, and anything special:

for f in $(grep -rl "api.openai.com\|ai.gateway" . | sort); do
  n=$(echo "$f" | sed 's|^\./||;s|/index.ts||')
  mdl=$(grep -ho "model: *['\"][^'\"]*['\"]" "$f" | head -1 | sed "s/model: *//;s/['\"]//g")
  json=$(grep -q "response_format" "$f" && echo json || echo -)
  img=$(grep -qE "image_url|modalities" "$f" && echo img || echo -)
  st=$(grep -q "stream: *true" "$f" && echo STREAM || echo -)
  printf "%-34s %-28s %s %s %s\n" "$n" "${mdl:-?}" "$json" "$img" "$st"
done

Flag these now, they change the plan:

• STREAM — the packaged gateway does not support streaming. Either extend it or leave that function out and say so explicitly.
• img — image input goes through callAIVision; image generation uses modalities/imageConfig.
• Audio / embeddings / TTS — not chat completions. They do not fit callAI as shipped; leave them and document the exception.

A real worked inventory: ../openrouter-gateway/assets/examples/inventario-funcoes.md.

Report the table to the user and agree on scope and order before migrating.

Step 2 — order the work

Simplest first, so the recipe is proven before it meets a hard case.

1. Pilot — one short function, no JSON mode, no images, easy to trigger from the UI and easy to eyeball the output.
2. Plain text batch — same shape, no surprises.
3. Functions with their own 429 handling — the manual retry must be removed; leaving it alongside the gateway's shows the user "rate limited" while the gateway is still retrying.
4. Long-form generation — check maxTokens; the gateway defaults to 1500, which truncates a page generator.
5. Special cases — vision, JSON mode, hot paths, public routes without a tenant.

Step 3 — migrate one function

Read ../openrouter-gateway/assets/examples/antes-depois.md — it has the real before/after for the three call patterns. Then, per function:

1. Capture the baseline. Trigger it with a real case and save the output. Without this there is nothing to compare against, and "it didn't crash" is not a quality check.
2. Replace the call:

import { callAI, AIGatewayError } from "../_shared/ai-gateway.ts";

const { content } = await callAI({
  functionName: "generate-summary",     // = folder name
  orgId,                                 // from body or JWT
  systemPrompt,
  userPrompt,
  maxTokens: 1500,                       // keep the value the function used
  temperature: 0.7,
  responseFormat: { type: "json_object" },  // only if it had response_format
});

3. Delete what the gateway now owns: the provider key lookup, the manual 429 branch, the data.choices[0] validation, the hardcoded model:.
4. Add the FUNCTION_DEFAULTS entry with primary + cross-provider fallback.
5. Translate the model name if it came from a provider-specific format: gpt-4.1-2025-04-14 → openai/gpt-4.1. Drop the date, prefix the provider, and confirm against https://openrouter.ai/api/v1/models.
6. Handle the error once, in the handler's catch:

} catch (error) {
  if (error instanceof AIGatewayError) {
    const status = { NO_API_KEY: 400, INVALID_KEY: 400, NO_CREDIT: 402,
                     RATE_LIMITED: 429, TIMEOUT: 504, UPSTREAM: 503 }[error.code] ?? 500;
    console.error(`[FN] ${error.code}`);          // the code, never the prompt
    return new Response(JSON.stringify({ success: false, error: error.message, code: error.code }),
      { status, headers: { ...corsHeaders, "Content-Type": "application/json" } });
  }
  …
}

7. Check and deploy:

deno check supabase/functions/<name>/index.ts
supabase functions deploy <name>

8. Smoke test with the same case as step 1. Three questions: is the output equivalent in quality (human review, not just "no error")? Did a row land in ai_usage_log with tokens > 0? Is latency comparable?

9. Commit alone:

refactor(ai): route <function> through the OpenRouter gateway

10. If the recipe did not match reality, fix the recipe before the next function. Fifteen more are about to follow it.

Step 4 — where orgId comes from

// 1. already in the body (most authenticated functions)
const { org_id: orgId } = await req.json();

// 2. derive from the JWT
const { data: { user } } = await supabase.auth.getUser(token);
const { data: m } = await supabase.from("org_members")
  .select("org_id").eq("user_id", user!.id).maybeSingle();
const orgId = m?.org_id;

// 3. public route with no session: omit it
//    → global defaults + platform key, logged with a null tenant

Omitting it on a public route is fine and deliberate — say so in a comment. What is not fine is accepting a tenant id from an unauthenticated client and treating it as trusted.

Step 5 — decommission the old provider

Only after every function is migrated and tested.

grep -rn "api.openai.com\|ai.gateway.lovable.dev\|OPENAI_API_KEY\|LOVABLE_API_KEY" \
  supabase/functions && echo "STILL REFERENCED" || echo "clean"

Then wait a week before removing the secret. If something needs a rollback, the key must still be there. After that:

supabase secrets unset OPENAI_API_KEY

Do not remove a secret still used for something that is not chat completions (embeddings, Whisper, TTS). Check before unsetting.

Step 6 — compare cost and quality

select function_name,
       count(*)                                as calls,
       round(avg(latency_ms))                  as avg_ms,
       sum(prompt_tokens + completion_tokens)  as tokens,
       round(sum(cost_usd)::numeric, 4)        as cost_usd,
       count(*) filter (where not success)     as failures,
       count(*) filter (where is_fallback)     as on_fallback
from ai_usage_log
where created_at > now() - interval '7 days'
group by 1 order by cost_usd desc nulls last;

• A function that jumped in cost → change its model in FUNCTION_DEFAULTS, do not go back to the old provider.
• on_fallback > 0 consistently → the primary model is broken; fix it, the fallback is not meant to be the normal path.

Rules for this run

• One function per commit. No bundled refactors, no "while I was in there".
• Never invent a model id. Confirm it in the OpenRouter catalog.
• Never paste a key, prompt or completion into a report.
• Deploys are outward actions. Ask before the first one of a session.
• A failed smoke test stops the batch. Report it; do not queue it as a TODO and keep migrating.