RossLabs-AI-Toolkit

Operational lessons for maintaining a Claude Code plugin marketplace. Load whenever making changes to `.claude-plugin/marketplace.json`, bumping a plugin version that's listed in a marketplace, adding/removing plugins from a marketplace, or troubleshooting "add marketplace" failures. Prevents schema rejection, stale manifests, CDN-lag false negatives, and install-command drift.

How to publish a public npm package to GitHub Packages for any project under the @tyroneross scope — Claude Code plugins, build-loop subsystems, prompt-builder, etc. Load whenever the user asks to "publish to GitHub Packages", "publish the plugin", "ship as an npm package", "make this installable", "release this to npm", or whenever bumping a package version that needs to ship. Covers single-package repos (build-loop pattern) and monorepo per-plugin publishing (RossLabs-AI-Toolkit pattern). Includes the GitHub Actions workflow template, the release ritual, and the verification checklist that catches the 403 / version-drift / scope-missing failures already seen in this account.

Use when building an evaluation harness for an LLM agent, RAG pipeline, or prompt — when the question is "is this actually working?" Encodes Anthropic and OpenAI's published 2025–2026 eval guidance — start with 20–50 real-failure tasks (not synthetic happy-paths), use three grader types in the right places (deterministic code / LLM judge / human review), measure pass@k vs pass^k for production reliability, grade outcomes not reasoning paths, calibrate the LLM judge against human review before trusting it, and detect eval saturation. Triggers on "build an eval", "build evals", "eval harness", "is this prompt working", "regression test for LLM", "how do I know my agent is better", "LLM judge", "pass@k", "compare two prompts on real data", "the prompt looks fine but production is broken". Vendor-neutral with starter templates.

Use when designing the surface of a tool that an LLM agent will call — function declarations, MCP server methods, OpenAI function-calling schemas, Anthropic tool_use definitions, or Agent SDK tools. Encodes Anthropic and OpenAI's published 2025–2026 guidance — namespaced names (service_resource_verb), semantically specific parameters (user_id over user), signal-dense return formats with a response_format knob, prompt-engineered error strings that guide the model toward the fix, and the anti-pattern of wrapping every API endpoint 1:1. Triggers on "design a tool for an agent", "write a function declaration", "MCP server tool", "function calling schema", "tool_use definition", "Agents SDK tool", "the agent keeps calling my tool wrong", "the agent loops on a tool error", "list-all is blowing up context", "tool descriptions". Cross-vendor; vendor wire-format notes in references/.

Use when building or operating a coding agent that has to work across multiple context windows or multiple sessions — when one shot won't fit. Encodes Anthropic's published 2025 harness pattern — initializer/coding split, claude-progress.txt + feature-list JSON + git commits as cross-context state, mandatory session-init protocol (read logs → review features → run e2e → start dev server), edit-status-only contract on the feature file. Triggers on "long-running agent", "multi-session agent", "context window keeps filling up", "agent forgets between sessions", "session handoff", "Codex to Claude Code handoff", "Claude Code to Codex handoff", "harness for an agent", "what does the agent read on session start". Bridges bookmark (session snapshots) and build-loop (single-session orchestration).

Use when designing a prompt or agent that will run at scale and you want to maximize prompt-cache hit rate. Encodes the cross-vendor invariant — order content static-to-dynamic so the cacheable prefix is stable — plus Anthropic's explicit cache_control breakpoint mechanics (4-breakpoint budget, 20-block lookback, 5m vs 1h TTL) and OpenAI's automatic prefix hashing (≥1024 token threshold, 128-token increments, 5–10min idle TTL, up to 24h on gpt-5.5+). Triggers on "prompt caching", "cache hit rate", "reduce LLM cost", "ephemeral cache", "cache_control", "prompt is slow on every call", "we're paying for the same tokens twice", "TTL", "cache breakpoint". Vendor-neutral spine plus per-vendor branches in references/.

Use when prompting a reasoning model — OpenAI o-series (o3, o4-mini, o5), GPT-5 with reasoning_effort, or Anthropic Claude with extended thinking enabled. This is a counter-skill — the default prompt-builder advice (CoT, role-priming, "think step by step") actively hurts reasoning models. Encodes OpenAI and Anthropic's published 2025–2026 guidance — zero-shot first, do NOT add chain-of-thought, do NOT role-prime, developer messages replace system, reasoning_effort is a tuning knob not a quality lever, "Formatting re-enabled" recovers structured output, preserve thinking blocks across tool turns on Anthropic. Triggers on "prompt an o-series model", "prompt o3 / o4-mini / o5", "GPT-5 reasoning_effort", "extended thinking on Claude", "reasoning model", "the o-series is being weird", "the model ignores my CoT prompt", "thinking blocks", "should I use chain-of-thought here".

Use when writing a system prompt for an LLM response that must cite sources, label every number with origin, and stay in a consistent voice. Composes three reusable rule blocks (citation contract, number-labeling, two-register voice) onto a base prompt and gives you the contract tests to keep them from drifting. Triggers on "build a system prompt", "make it cite sources", "add citations", "ground the output", "label numbers", "truthfulness rule", "system prompt for RAG", "system prompt for an audit view", "stop the model from making up numbers", "the model is citing things that aren't in the retrieval list". Designed for RAG, agentic Q&A, recommendation surfaces, and audit/explain views. Not for creative writing, code generation, or pure structured-extraction tasks.

Use when an LLM workflow needs structured output AND calibrated scoring AND an auditable decision trail. Encodes the two-pass pattern - Pass 1 (cheap/fast model like Groq, Haiku, or gpt-4o-mini) emits coarse structure with sentinel defaults for fields the second pass will fill; Pass 2 (precision model like Opus, gpt-4o, or Claude Sonnet) fills the scored fields against a strict Zod schema; a deterministic TS post-process computes the final ranking formula. Triggers on "two-phase LLM", "two-pass LLM", "decompose then score", "classify then rank", "structured output with audit", "auditable LLM workflow", "sentinel defaults", "deterministic post-process", "the LLM scoring is opaque", "I need to show my work", "the model keeps drifting on the ranking formula". Designed for workflow decomposition, multi-criteria scoring, classify-then-rank tasks. Not for single-shot completion, conversational chat, or low-stakes ranking where one frontier-model call is good enough.

Generate a living, LLM-navigable PRD for an app or feature by answering 3-5 strategic questions. Output is a generative model an AI coding agent can reason from to make tactical decisions without escalating to the human on every choice. Use when starting a new app, scoping a new feature surface, doing a mid-project realignment after reactive iteration, before a major pivot, or whenever an LLM keeps asking the same kind of "should I do A or B?" question because the strategic frame is missing. Trigger phrases include "I want to build an app for", "scoping a new project", "thinking about building", "PRD for", "what should this product be", "starting a new app", "we've been building reactively", "the LLM keeps asking the same kind of question". Skill produces a markdown PRD with frontmatter (always-true principles + load_when triggers), an LLM Navigation Map, a Section Index for offset/limit reads, and a Fidelity check that validates the PRD has enough density to derive any tactical decision.

Score a single artifact (skill, feedback note, agent definition, research entry, prompt, or any markdown deliverable) against a locked 5-dimension rubric with evidence anchoring. Use when the user asks to "judge X", "score X", "rubric-score X", "evaluate X against criteria", or as a quality gate before promoting a skill/agent. Returns a deterministic table (dimension, score 1-5, evidence quote, one-line rationale) and an aggregate score. No Chain-of-Thought, no freeform critique — the rubric carries the structural work.

Apply the Secrets Vault contract to any MCP server you build, edit, or review. The contract is metadata-only — tools return references and IDs, never secret values, bearer tokens, or internal state. Activates when working on `mcp__*` tools, `.mcp.json`, or any MCP server source file (`server.py`, `server.ts`, `*.mcp.*`).

Routes "what should I test first" questions to substrate-specific defaults wrapped in universal risk-based prioritization. Use when the user asks to "test", "verify", "validate" something, says "let's check this", "how do I test", "what should I test first", or after substantial implementation work that needs verification. Detects app type (web UI, iOS/macOS, plugin, MCP server, CLI, LLM-driven, deploy/infra) from project signals and returns a 3-step priority-ordered test sequence backed by the testing-prioritization research entry.

Use when the user asks to "sync plugins", "check plugin versions", "which plugins are drifting", "refresh marketplace versions", "update the plugin readme", "install plugin-sync hooks", "lint my plugin manifests", "validate plugin.json paths", "audit Codex installability", "plugin-sync status", or similar. Tool tracks local Claude Code plugins across source repos, marketplace manifests, and Claude Code's installed_plugins.json registry; detects drift and optionally fixes it; also lints plugin.json path fields against the manifest reference rules and audits additive Codex packaging. Personal tool scoped to its author's machine — bail silently if ~/.config/claude-plugins/config.json is missing.

Design, evaluate, and improve agentic harnesses for developer tools, assistants, workflow runtimes, copilots, and AI-powered products — including agents built on local or open-source models (Ollama, llama.cpp, vLLM, Llama, Qwen, DeepSeek, Mistral). Use when work involves tool-use architecture, permissions, approval gates, workflow state, durability, context and memory systems, evaluation strategy, observability, operator visibility, framework selection (LangGraph, CrewAI, Pydantic AI, smolagents, DSPy, AutoGen, DeepAgents), memory substrate choice, or phased implementation plans for an AI system. Trigger when symptoms imply harness gaps too — stale context, surprising tool calls, sessions that die on crash, missing approval controls, costs spiraling without clear visibility, tool counts crossing ~50, context windows routinely hitting 92%+ capacity, local-model agents hallucinating tool calls, or on-device agents failing silently after model swaps.

Use when the user wants to analyze prior Codex archived sessions to infer request, authorization, verification, and handoff patterns and use those patterns to improve future build workflows.

Autonomous metric-driven optimization loop. Constrained scope + mechanical metric + git commit/revert + fast iteration. Use for build optimization, test coverage, prompt tuning, doc quality, or runtime performance.

Use for significant multi-step research-and-planning work that benefits from a phase-based loop, orchestration, parallel subagents, validation, iteration, and context continuity.

Use when resuming work in a project that may already have a build-loop-auto-research context snapshot or registry fallback trailhead.

Use when work is about to span sessions, risk compaction, or switch contexts and you want a bookmark-style local snapshot of the current project state.

Use before repeating an investigation, debugging loop, or multi-iteration fix so Codex can check modular lessons learned and decide whether a known fix or likely match already exists.

Use when the user has a rough request, draft plan, or prompt that needs to be tightened into a clearer, simpler, more verifiable build brief.

Use when a user wants a research-backed build packet, a concise build investigation, a history-pattern analysis, or a brief/prompt optimization pass.

Use when the user wants a research packet plus a decision-complete implementation plan for a product, feature, algorithm, prompt, bugfix, or refactor, but does not need the full explicit phase-by-phase build loop narration.

Design, evaluate, and improve agentic harnesses — the orchestration layer around LLM-powered tools, agents, assistants, copilots, workflow runtimes, and AI-driven product features. Use this skill whenever the user mentions building an agentic system, structuring tool use, adding permissions or approval gates, designing multi-step AI workflows, managing context windows or memory, making agents durable or resumable, evaluating or pressure-testing an existing harness, planning phased implementation for an AI product, reviewing agent architecture, improving agent UX or observability, choosing between frameworks (LangGraph, CrewAI, Pydantic AI, smolagents, DSPy, AutoGen, DeepAgents), picking a memory substrate, or asking how to know if their harness is actually good. Also use when the user describes problems that imply harness gaps — agents doing unexpected things, context getting stale, sessions not surviving crashes, tools running without permission, or costs spiraling — even if they do not use the word "harness

Designs, evaluates, and improves agentic harnesses for developer tools, assistants, workflow runtimes, copilots, and AI-powered products — including agents built on local and open-source models. Applies when work involves defining or reviewing tool-use architecture, permissions, workflow state, durability, context and memory systems, evaluation strategy, observability, user experience, framework selection, or phased implementation plans for an agentic system.

This skill activates when the user asks to "scan architecture", "what's my stack", "check dependencies", "outdated packages", "show project structure", "what components do I have", "refresh architecture", "scan project", "health check", "vulnerability scan", "what changed architecturally", "is my architecture data stale", or when starting a session where architecture data may be outdated. Provides architecture scanning, status checking, and health analysis via NavGator MCP tools.

This skill activates when the user mentions "what was I working on", "continue from last session", "restore context", "what did we decide", "pick up where I left off", "compaction happened", "lost context", "session context", "resume work", "what's the status", "where did we leave off", "prior session", "context lost", or when resuming work after a break. Provides session continuity by accessing bookmark snapshots via MCP tools.

This skill should be used when the user asks to "debug this", "fix this bug", "why is this failing", "investigate error", "getting an error", "exception thrown", "crash", "not working", "what's causing this", "root cause", "diagnose this issue", or describes any software bug or error. Also activates when spawning subagents for debugging tasks, using Task tool for bug investigation, or coordinating multiple agents on a debugging problem. Provides memory-first debugging workflow that checks past incidents before investigating.

This skill should be used when the user asks to "build this UI", "implement this design", "make it look like", "check my progress", "does this match what I described", "build this component", "make this work", "implement the layout", "set up this page", or when frontend files (.tsx, .jsx, .vue, .svelte, .css, .scss, .swift) are being actively edited and implementation needs to stay aligned with user intent.

Suggest capturing screenshots or video at opportune moments during development