speq-app-builder

name	speq-app-builder
description	Use this skill when building an application, CLI, or demo that consumes the speq library to evaluate prompts against AI SDK language models and present the judged results.

speq App Builder

This skill is for building programs that use speq.

Use it when the task involves:

• creating a CLI, web app, script, or service that depends on speq
• wiring AI SDK LanguageModel instances into speq
• defining LLMSpec[] inputs and rendering judged results
• choosing where to use formatSpeqTable versus a custom UI
• adding provider packages to the consuming app without changing packages/speq

What speq Expects

speq exposes three main entry points:

• defineConfig(...)
• runSpeq(...)
• formatSpeqTable(...)

The consuming app is responsible for:

• choosing and installing an AI SDK provider package such as @ai-sdk/openai
• creating LanguageModel instances
• providing LLMSpec[]
• deciding how to display or persist the returned TestResult[]

Default Integration Pattern

1. Install speq and the AI SDK provider package in the consuming app.
2. Create one or more AI SDK LanguageModel instances.
3. Build a SpeqConfig with defineConfig.
4. Define LLMSpec[] close to the app logic or load them from the app's own config layer.
5. Call runSpeq({ config, specs }).
6. Render with formatSpeqTable(results) for terminal output, or build a custom UI from TestResult[].

Example Shape

import { openai } from "@ai-sdk/openai";
import { defineConfig, formatSpeqTable, runSpeq } from "speq";

const config = defineConfig({
  models: [
    { name: "gpt-4.1-mini", model: openai("gpt-4.1-mini") },
  ],
});

const specs = [
  {
    prompt: "PythonとGoの違いを説明して",
    expects: {
      language: "same as input",
      maxLength: 200,
      format: "bullet",
    },
  },
];

const results = await runSpeq({ config, specs });
console.log(formatSpeqTable(results));

When Building a Consuming App

• Keep provider-specific setup in the consuming app, not in packages/speq.
• Prefer giving each model a stable display name such as openai:gpt-4.1-mini.
• If the app needs richer output, use TestResult[] directly instead of parsing the table string.
• If the app is interactive, cache or store specs in the app layer rather than extending speq with file loading.

UI Guidance

For terminal tools:

• use formatSpeqTable first
• add raw output sections only when users need to inspect failures

For web apps or dashboards:

• map TestResult[] into cards or tables
• surface judgement.overall prominently
• expose output and error.message for failed rows

Validation

When you change a consuming app inside this monorepo, verify from the root:

bun run check
bun run demo

If the change also touches speq, inspect:

• packages/speq/src/index.ts
• packages/speq/src/types.ts
• README.md

Done Criteria

Before finishing, aim to have:

• the consuming app importing speq cleanly
• provider setup contained in the app, not the library
• runSpeq wired with real or mock models
• judged results rendered in a form appropriate for the app