harness-sdk-protocol

name	harness-sdk-protocol
description	Use for reusable harness sdk protocol recipes in software and coding-agent work.

Harness, SDK & wire protocol — recipes (generic tier)

On-demand recipes (3). Trigger→action heuristics; pull the relevant one when its situation arises.

1. Shadow-run an LLM controller and cap its call cadence before granting live control; verify every producing state calls the hook

generic · ⚠ session-derived, unverified

Before granting an LLM controller live authority in any agentic system, shadow-run its full decision loop (build context -> ask model -> validate -> log accept/reject -> compare to the existing deterministic baseline -> aggregate metrics) WITHOUT letting it take action, to confirm it picks legal/valid/safe choices. Add per-controller cadence control (fire the model only every N steps) FIRST: an unthrottled heuristic shadow produced 8,000+ decisions in one short run, and a real model on that path would issue thousands of API calls. NEGATIVE: 'wire the LLM in' can be a misnomer -- an action the LLM is supposed to own may still be decided deterministically (an executor translated intents into actions, but the controller never asked the LLM and made them itself). When delegating a decision class to an LLM, check EVERY state that can produce that action, not just the obvious entry point, and verify the hook actually fires with a trace; triage formatting glitches (malformed/concatenated output) separately from call success -- they are a prompt/sanitization bug distinct from whether the call worked. _{sources: codex:019e131a-1bb0-7400-b54a-f208202b291d, claude-code:aae25940-ade3-41bc-b817-517dc86ccebf, codex:019e14a5-e574-7770-953b-17019f2f5e74}

2. Build one env-driven LLM provider abstraction; force structured output via per-kind tool definitions; use inference-profile model IDs

generic · tool: bedrock-claude-provider

Factor the LLM provider behind an interface early so models/providers swap without touching game logic (a monolithic ~4720-line bot was hard to evolve; its successor split into chat/llm/llm_dispatch/llm_provider/types and generalized a single OpenAI client into a multi-provider Anthropic/OpenAI/Bedrock one). Build one provider abstraction reused across all loop consumers (per-game advisor, post-game analyst, code-evolution editor): AWS Bedrock primary via boto3 converse() (or stdlib HTTPS+SigV4 to avoid third-party SDKs), OpenRouter/direct-Anthropic fallbacks, selected from env vars plus a 'provider:model' spec, and print the resolved provider+model at startup so the operator confirms the active path. AWS Bedrock REJECTS direct on-demand invocation of newer Claude models ('Invocation of model ID ... with on-demand throughput isn't supported. Retry with an inference profile.') -- use the inference-profile form (us.anthropic.* or global.anthropic.* prefix, discoverable via bedrock.list_inference_profiles()), not the bare anthropic.* ID. Prefer provider-level structured output (register one output schema per call kind as a provider tool/function definition with tool_choice set to it, translating one Anthropic input_schema into the OpenAI function.parameters shape) over schema-in-prompt, keeping prompt-embedded hints only as fallback. Implement the converse tool-use loop as append-assistant-content -> execute tool calls -> append results as a user turn -> repeat to a max_rounds cap, returning token-usage totals for cost budgeting. Default the small game-control model to claude-haiku-4-5 (temp 0.2, max ~512 tokens, ~12s timeout), honoring fallback env vars COGAMES_LLM_MODEL/ANTHROPIC_SMALL_FAST_MODEL/ANTHROPIC_MODEL. Verify platform capability claims against docs/code before designing around an assumed limitation (an agent wrongly assumed LLMs couldn't run in the cogames tournament; API keys CAN be passed as runtime secrets so in-process LLM calls ARE supported). For vision-language calls a 1x1 PNG was rejected ('Could not process image') while 64x64 succeeded, and cache paid vision-inference on a content key (frame plus request) so reruns with a fresh run_id reuse prior labels -- spend-idempotency keys to inputs, not run identity. When a compiled binary must call Bedrock but should never see AWS creds, shell out to 'aws bedrock-runtime invoke-model' (uses the boto3 chain) instead of embedding an SDK. _{sources: opencode:ses_219efd206ffe6azQSiSZsvnxX4, opencode:ses_21e58aef7ffeoJ7uZeDgGBCfIh, opencode:ses_21f89f53effe65xlVIKX1hvVv3, codex:019e13e1-e883-7fe0-93b4-54adeac75285 (+6)}

3. Invoke Claude Code and Codex non-interactively from scripts

generic

To invoke Claude Code non-interactively, run claude -p "<prompt>" (processes the prompt, prints to stdout, exits); add --allowedTools "Read,Edit,Bash" to pre-approve tools (granular patterns like "Bash(git *)"), --add-dir <path> for access outside the cwd, --bare to skip hooks/plugins/CLAUDE.md/memory for deterministic side-effect-free runs, and --output-format json for structured output (result, session_id, usage, total_cost_usd); piped stdin is capped at 10MB. To invoke Codex non-interactively, use codex exec "<prompt>" (alias e): set the working dir with -C <dir>, write permissions with -s <mode> (read-only default / workspace-write / danger-full-access), --skip-git-repo-check (Codex requires a git repo by default), --ephemeral to avoid persisting sessions; the final message goes to stdout and progress to stderr, -o <file> writes the final message, --json gives a JSONL event stream. NEGATIVE: when Codex is authenticated via a ChatGPT account rather than an API key, model selection is limited to the account tier and requesting an unsupported model (e.g. o4-mini) returns 400; supplying CODEX_API_KEY unlocks the full roster. _{sources: players_checkouts/players/tools/cogbase/docs/cli_runners.md}