vidai-mock

name	vidai-mock
description	Use vidai-mock as a local LLM inference simulator for developing and testing software that targets OpenAI-, Anthropic-, and compatible chat APIs. Covers startup, provider/template configuration, streaming physics, tool-call simulation, chaos testing, and observability-driven validation.

Vidai Mock Skill

When To Use This Skill

Use this skill when you need a local, offline mock server that behaves like real LLM providers and supports:

• API-compatible chat/completions style testing.
• Streaming behavior (TTFT, token pacing, jitter).
• Tool/function call payload simulation.
• Fault injection for retry, timeout, and parser hardening tests.
• RAG citation and structured-output test scenarios.

Level 1: Critical Path (Start Here)

Follow this path before adding advanced behavior.

1. Confirm the binary is available.

command -v vidaimock

1. Start with default settings, then verify the server is reachable.

vidaimock
curl -sS http://localhost:8100/metrics | head

1. Point your SDK to the local base URL.

# OpenAI-style client example
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8100/v1", api_key="sk-local-test")

# Anthropic-style client example
from anthropic import Anthropic

client = Anthropic(base_url="http://localhost:8100/v1", api_key="sk-local-test")

1. Send a baseline request before adding chaos/streaming overrides.

curl http://localhost:8100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "hello"}],
    "stream": false
  }'

If your client requires an API key field, pass any non-empty string.

Level 2: Daily Development Workflow

Configuration Precedence

Use this order when changing behavior:

1. CLI flags.
2. Environment variables.
3. mock-server.toml.
4. Built-in defaults.

Typical controls:

• Host/port.
• Worker count.
• Logging level.
• Global latency and chaos defaults.

Workspace Layout

Keep simulation assets close to the app under test.

.
|- mock-server.toml
`- config/
   |- providers/
   |  |- openai.yaml
   |  `- custom-provider.yaml
   `- templates/
      |- openai/
      `- custom/

Provider Routing Model

Each provider entry should define:

• A unique name.
• A matcher regex for request paths.
• Optional priority when multiple routes overlap.
• Optional request_mapping to normalize input fields.
• Exactly one response source:
  • response_template for file-based templates.
  • response_body for inline templates.

Use request_mapping to normalize different inbound schemas into a stable template context.

Template Runtime Model

Templates run with request-scoped context values such as:

• Parsed JSON body.
• Request headers.
• Query parameters.
• HTTP method and path.

Use helper functions for realistic payloads (for example: generated IDs, timestamps, bounded random values).

Level 3: Scenario Playbooks

Streaming Physics

For streaming requests ("stream": true), design templates around lifecycle phases:

1. Start event.
2. Chunk loop.
3. Stop event.

Use a chunk placeholder during the loop phase to emit incremental token-like packets. Validate behavior with unbuffered curl:

curl -N http://localhost:8100/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "count to five"}],
    "stream": true
  }'

Tool And Function Calling

To simulate tool usage, your assistant message must emit the provider-specific tool-call structure, including stable function names and arguments your app expects.

Recommended pattern:

1. Route tool-capable endpoints with a dedicated provider matcher.
2. Use template logic against the last user message to choose which tool to call.
3. Return a completion state that indicates tool invocation rather than plain text finalization.

This is where most parser or integration regressions happen; keep at least one golden-path and one malformed-path test.

JSON/Structured Output

For structured-output tests, return content that is valid JSON in success paths and intentionally invalid JSON in negative paths. This validates both your strict parser and your fallback handling.

Chaos Injection For Resilience

Use per-request headers to inject failures deterministically during test runs.

# Force a 500 response
curl -H "X-Vidai-Chaos-Drop: 100" http://localhost:8100/v1/chat/completions -d '{"model":"gpt-4"}'

# Force malformed JSON
curl -H "X-Vidai-Chaos-Malformed: 100" \\
  http://localhost:8100/v1/chat/completions -d '{"model":"gpt-4"}'

# Add 2s TTFT and jitter
curl -H "X-Vidai-Latency: 2000" -H "X-Vidai-Jitter: 0.2" \\
  http://localhost:8100/v1/chat/completions -d '{"model":"gpt-4"}'

# Slow chunk cadence (streaming)
curl -H "X-Vidai-Chaos-Trickle: 1000" \\
  http://localhost:8100/v1/chat/completions -d '{"model":"gpt-4","stream":true}'

Prefer request-level chaos controls in tests so failures are explicit and reproducible.

RAG And Citation Validation

Use template variants to test:

• Citation markers in generated text.
• Citation metadata blocks in payloads.
• Retrieval-latency effects before first token.
• Hallucinated or non-existent references for guardrail tests.

Create separate endpoints (or provider matchers) for trusted and adversarial RAG scenarios.

Observability And Diagnostics

Metrics

Scrape metrics from:

GET http://localhost:8100/metrics

Track at least:

• Request volume by route and status.
• Request latency distribution.

Logs

Control verbosity with RUST_LOG or config log level.

RUST_LOG=debug vidaimock

Use debug during template/protocol work and reduce to info for routine CI runs.

Agent Operating Checklist

1. Start with a no-chaos baseline request and assert schema compatibility.
2. Enable streaming and validate chunk assembly logic.
3. Enable tool-call response shapes and validate tool dispatch parsing.
4. Add structured-output success and failure cases.
5. Add resilience tests using deterministic chaos headers.
6. Add RAG citation positive and negative cases.
7. Verify metrics/log signals for each scenario.

Troubleshooting

• No response from client:
  • Check base URL and path compatibility (/v1/...).
  • Confirm the server process is running and bound to expected host/port.
• Unexpected template output:
  • Verify route matcher regex and provider priority.
  • Confirm mapped fields exist in incoming request shape.
• Streaming does not appear incremental:
  • Confirm "stream": true in request body.
  • Use curl -N or a client that does not buffer SSE chunks.
• Tool-call parser failures:
  • Recheck provider-specific tool schema and finish-state semantics.
  • Validate argument encoding expectations in your app.

Use this skill as the default local harness for LLM integration tests before spending budget on external provider calls.