llm-api-contract

name	llm-api-contract
description	LLMService API contract — the correct signatures for complete(), stream(), embed(), and response types. Use when calling LLMService from any layer, writing reasoning strategies, or building LLM-dependent features.
user-invocable	false

LLMService API Contract

The most common source of bugs in this codebase is calling LLMService incorrectly. This skill defines the exact API contract.

Service Definition

export class LLMService extends Context.Tag("LLMService")<
  LLMService,
  {
    readonly complete: (
      request: CompletionRequest,
    ) => Effect.Effect<CompletionResponse, LLMErrors>;
    readonly stream: (
      request: CompletionRequest,
    ) => Effect.Effect<Stream.Stream<StreamEvent, LLMErrors>, LLMErrors>;
    readonly completeStructured: <A>(
      request: StructuredCompletionRequest<A>,
    ) => Effect.Effect<A, LLMErrors>;
    readonly embed: (
      texts: readonly string[],
      model?: string,
    ) => Effect.Effect<readonly number[][], LLMErrors>;
    readonly countTokens: (
      messages: readonly LLMMessage[],
    ) => Effect.Effect<number, LLMErrors>;
    readonly getModelConfig: () => Effect.Effect<ModelConfig, never>;
  }
>() {}

CompletionRequest — What You Send

// Matches packages/llm-provider/src/types.ts — keep in sync
export type CompletionRequest = {
  readonly messages: readonly LLMMessage[];
  readonly model?: ModelConfig;
  readonly maxTokens?: number;
  readonly temperature?: number;
  readonly stopSequences?: readonly string[];
  readonly tools?: readonly ToolDefinition[];
  readonly systemPrompt?: string;
  readonly logprobs?: boolean;
  readonly topLogprobs?: number;
};

CORRECT usage:

const response =
  yield *
  llm.complete({
    messages: [{ role: "user", content: "What is quantum computing?" }],
    systemPrompt: "You are a helpful assistant.",
    maxTokens: 300,
    temperature: 0.7,
  });

WRONG — these will NOT compile:

// ❌ WRONG: No "prompt" field exists
llm.complete({ prompt: "Hello" });

// ❌ WRONG: No "input" field exists
llm.complete({ input: "Hello" });

// ❌ WRONG: messages must be an array of LLMMessage
llm.complete({ messages: "Hello" });

// ❌ WRONG: Do NOT wrap in Effect.tryPromise — complete() already returns Effect
yield* Effect.tryPromise({ try: () => llm.complete({ ... }), ... });

CompletionResponse — What You Get Back

interface CompletionResponse {
  readonly content: string; // ← The text response (NOT .text, NOT .result)
  readonly stopReason: StopReason; // "end_turn" | "max_tokens" | "stop_sequence" | "tool_use"
  readonly usage: TokenUsage; // Token accounting
  readonly model: string; // Model that was used
  readonly toolCalls?: readonly ToolCall[]; // If tools were invoked
}

interface TokenUsage {
  readonly inputTokens: number;
  readonly outputTokens: number;
  readonly totalTokens: number;
  readonly estimatedCost: number; // USD (NOT .cost, use .estimatedCost)
}

CORRECT field access:

const response = yield* llm.complete({ messages: [...] });
const text = response.content;                    // ✅ .content
const tokens = response.usage.totalTokens;        // ✅ .usage.totalTokens
const cost = response.usage.estimatedCost;        // ✅ .usage.estimatedCost
const reason = response.stopReason;               // ✅ .stopReason

WRONG field access:

// ❌ response.text — does not exist, use .content
// ❌ response.result — does not exist, use .content
// ❌ response.usage.cost — does not exist, use .usage.estimatedCost
// ❌ response.usage.confidence — does not exist
// ❌ response.output — does not exist, use .content

LLMMessage Types

type LLMMessage =
  | { readonly role: "system"; readonly content: string }
  | {
      readonly role: "user";
      readonly content: string | readonly ContentBlock[];
    }
  | {
      readonly role: "assistant";
      readonly content: string | readonly ContentBlock[];
    };

Error Handling

LLMService methods return Effect.Effect<T, LLMErrors> — they already handle errors using Effect. Do NOT wrap calls in Effect.tryPromise. Use Effect.mapError or Effect.catchTag to transform errors:

// ✅ CORRECT: pipe with mapError
const result = yield* llm.complete({ messages: [...] }).pipe(
  Effect.mapError((e) => new MyError({ message: `LLM failed: ${e.message}` })),
);

// ✅ CORRECT: catch specific error tags
const result = yield* llm.complete({ messages: [...] }).pipe(
  Effect.catchTag("LLMRateLimitError", (e) =>
    Effect.sleep(e.retryAfterMs).pipe(Effect.flatMap(() => llm.complete({ messages: [...] }))),
  ),
);

Embeddings (Tier 2 Memory Only)

// embed() is the SOLE embedding source for the entire framework
// Memory Tier 1 does NOT call embed()
// Memory Tier 2 calls embed() for sqlite-vec KNN search
const vectors = yield * llm.embed(["text to embed", "another text"]);
// Returns: readonly number[][] (one vector per input text)

Prompt Caching (Anthropic)

import { makeCacheable } from "@reactive-agents/llm-provider";

// Wrap static content in cacheable blocks for Anthropic prompt caching
const message: LLMMessage = {
  role: "user",
  content: [
    makeCacheable(staticSystemContext), // Cached across requests
    { type: "text", text: dynamicInput },
  ],
};

Model Presets

Available presets: "claude-haiku", "claude-sonnet", "claude-sonnet-4-5", "claude-opus", "gpt-4o-mini", "gpt-4o"

import { ModelPresets } from "@reactive-agents/llm-provider";
const config = ModelPresets["claude-sonnet"]; // { provider, model, costPer1MInput, ... }

Provider Fallback Chain

// FallbackChain in @reactive-agents/llm-provider
// Automatically falls back to secondary providers on error
.withFallbacks({
  providers: ["anthropic", "openai"],
})

Test Provider

The test provider is auto-selected when using .withTestScenario(). It returns deterministic responses from the TestTurn[] array.