inference-smoke-tests

name	inference-smoke-tests
description	Run repeatable inference smoke tests using geppetto/pinocchio example binaries (single-pass, streaming, tool-loop, OpenAI Responses thinking) including tmux-driven TUI tests. Use when refactors touch InferenceState/Session/EngineBuilder, tool calling loop, event sinks, provider request formatting, or when you need a quick 'does inference still work?' checklist.

Inference Smoke Tests

Quick Start (Recommended)

Run the fast suite (geppetto non-TUI + pinocchio agent TUI) via the bundled script:

bash geppetto/.codex/skills/inference-smoke-tests/scripts/run_smoke.sh --quick

If you need the full manual checklist, open:

geppetto/.codex/skills/inference-smoke-tests/references/playbook.md

Preconditions

• Ensure OPENAI_API_KEY is set (for OpenAI Chat + OpenAI Responses).
• Ensure Claude credentials are available (e.g. ANTHROPIC_API_KEY) if you want the Claude tool-calling smoke step to pass.
• Ensure tmux is installed (required for non-interactive TUI runs).
• Expect costs: these tests make real API calls.

Workflow Decision Tree

1. Validate provider “thinking” streaming (Responses)?

• Run geppetto/cmd/examples/openai-tools in --mode thinking.

2. Validate tool loop orchestration?

• Run geppetto/cmd/examples/generic-tool-calling.

3. Validate Bubble Tea TUI event flow (thinking deltas + final)?

• Run pinocchio/cmd/agents/simple-chat-agent in tmux.

4. Validate Claude tool calling?

• Run geppetto/cmd/examples/claude-tools with --ai-api-type claude --ai-engine claude-haiku-4-5.

5. Validate multi-turn chat state persistence?

• Run pinocchio TUI chat in tmux (manual) and/or pinocchio webchat in browser (manual).

What “Benefits From InferenceState” (Rules of Thumb)

Already benefits (multi-turn, cancel-sensitive, tool-loop, strict provider validation):

• pinocchio TUI chat (pinocchio/cmd/pinocchio … --chat)
• pinocchio agent TUI (pinocchio/cmd/agents/simple-chat-agent …)
• pinocchio webchat (pinocchio/cmd/web-chat)
• geppetto example runners that execute via geppetto/pkg/inference/core.Session

Could benefit (optional; mainly consistency/cancel):

• pinocchio/cmd/examples/simple-redis-streaming-inference (transport-focused; currently eng.RunInference direct)
• pinocchio/cmd/examples/simple-chat (exercises PinocchioCommand runner; could benefit indirectly if that runner standardizes on InferenceState)

Does not apply (not an inference runner):

• geppetto/cmd/examples/citations-event-stream

Troubleshooting (Common Failure Modes)

“OpenAI Responses 400” errors

• Re-run with higher logging:
  • Add --log-level debug --with-caller where supported.
• Confirm you’re using the correct provider mode:
  • --ai-api-type openai-responses
• If the error mentions invalid parameter support (e.g., temperature unsupported), it’s model-dependent; reduce parameters and retry.

TUI doesn’t submit the prompt

• Some TUIs submit on Tab (not Enter).
• Always capture logs to a file and confirm inference actually ran (look for EventPartialCompletionStart, EventFinal).

References

When you need copy/paste commands for the full sweep, read:

• geppetto/.codex/skills/inference-smoke-tests/references/playbook.md

When you need to find new example entry points, search:

rg -n "cmd/examples" -S geppetto/cmd/examples pinocchio/cmd/examples
rg -n "cmd/agents" -S pinocchio/cmd/agents