| name | safe-agent-builder |
| description | Scaffolds a NEW AgentStack agent with a DETERMINISTIC quality gate and anti-hallucination input-guard baked in by default — not the prompt-only guardrails that LLMs ignore. Builds the full flow (kb_search → processor → function-validator → condition-gate → pass/error emitters → terminal message nodes), wires the KB, and smoke-tests BOTH the happy path and a deliberately bad input to prove the gate fails closed. Use this whenever the user wants to "create an agent", "scaffold an agent", "build a new agent", "napravi/kreiraj agenta", "dodaj agenta u pipeline", "set up an AgentStack agent", or wants an agent that "won't hallucinate", "blocks bad output", "has a quality gate", or "is production-safe" — even if they don't say "deterministic" or "gate". Prefer this over plain scaffolding when safety, anti-hallucination, or fail-closed behavior matters. Do NOT use for debugging an existing agent (use a debugger skill) or editing one field (use as_patch_node_field directly).
|
Safe Agent Builder
Why this skill exists (read this first — it's the whole point)
Most agent scaffolding puts quality rules in the prompt: "NEVER fabricate stats", "emit
MALFORMED_PAYLOAD if input is missing". LLMs ignore these when they 'want to help.' Proven the
hard way: given an empty brief, a Hook Writer agent invented a fake trend and produced 5 posts
that passed every prompt-level check and reached the review queue. The prompt said "block bad
input" — the model fabricated instead.
The fix is deterministic enforcement: a function node (real code, runs every time) decides
PASS/BLOCK, a condition node routes, and a terminal message node surfaces the verdict. Code
doesn't get talked out of its rules. This skill builds that structure by default.
The four non-negotiable rules (everything below serves these)
- Quality enforcement lives in a
function validator, never only in the prompt. The prompt
guides; the validator decides. If a rule matters, it must be checkable in code.
- Every branch ends in a
message node. Only ai_response, message, and call_agent
nodes emit output; function and condition emit nothing. Whatever runs last and emits is
what surfaces. A branch ending on a function node silently leaks the raw upstream output.
- Guard the INPUT, not just the output. Gates catch bad format and fabricated numbers but
not a fabricated topic. So the validator must reject an empty / "N/A" / missing core input and
a wrong item count — otherwise the LLM invents one and everything downstream faithfully spreads it.
- Fail closed. A BLOCKED payload must lack the shape a downstream consumer saves/acts on, so a
block is inert by construction (not "saved anyway").
The standard safe-agent flow
kb_search ─→ [web_search?] ─→ processor (ai_response, → payload var)
│
validator (function: parse + input-guard + format + banned + stat-trace)
│ returns "PASS" or JSON [{rule,detail,severity:"error"|"warning"}]
│
gate (condition: validator_result == "PASS")
┌───────────┴────────────┐
PASS │ │ ELSE
pass-emitter (function: error-emitter (function:
compute warning flags) {status:"BLOCKED",violations,warnings})
│ │
[call_agent → downstream?] │
│ │
pass-msg (message {{final}}) fail-msg (message {{error}})
Both leaves are message nodes (rule 2). On PASS with a downstream agent, the call_agent sits
between the emitter and pass-msg so the deepest agent's result bubbles up last.
Workflow
Set up a task list, then:
1 — Collect the spec
Ask (one AskUserQuestion call): pipeline role (standalone / receives / sends / middle), model
(default claude-sonnet-4-6), web search (yes/no), and the core input field the agent must
receive (e.g. trend, ticket, lead) — this is what the input-guard protects. Derive a slug.
2 — Generate the processor prompt
Single ai_response processor. The prompt still states the quality rules (it helps the model aim
right) AND defines the exact output shape. But it does NOT carry enforcement alone — the validator
does. Keep CONFIDENCE + DATE conventions. Read references/flow-templates.md → "Processor prompt".
3 — Build the flow WITH the gate
This is the differentiator. Read references/flow-templates.md for the exact node + edge JSON and
the validator code template (parametrized by the core field, item count, banned list). Build:
kb_search → [web_search] → processor → validator → gate → {pass-emitter → [call_agent] → pass-msg}
/ {error-emitter → fail-msg}. Use as_update_flow (structural). Back up first with
as_inspect_flow.
4 — Verify the structure (mandatory, grounded)
Re-as_inspect_flow. Confirm: validator code contains the input-guard (missing_<field> +
wrong_count), the gate routes on == "PASS", BOTH branches terminate in a message node, edge
count is right. If anything is off, STOP and fix — do not proceed.
5 — Wire the KB
Create the KB (UI or script), seed it, as_patch_node_field the kb_search knowledgeBaseId,
poll embedding to ready. (KB lives at /knowledge/<agentId>, not under "Memory".)
6 — Smoke-test BOTH cases (this is what makes it trustworthy)
- Happy path: a valid input → expect PASS → output surfaces (or downstream runs).
- Bad input (the critical one): send the empty / "N/A" / malformed input → expect a clean
BLOCKED from THIS agent's
fail-msg (look for the agent's own status:"BLOCKED" + your
missing_<field>/wrong_count rule), NOT fabricated content and NOT a leaked raw payload.
If the bad input produces content → the guard isn't wired right; fix before declaring done.
- If the happy path BLOCKS unexpectedly (a valid input that should pass gets blocked): suspect a
validator reading an object/array variable — e.g. a
web_search output — with String() instead
of JSON.stringify. It silently becomes "[object Object]" and any grounding/substring check
false-blocks 100% of the time. Add a temp debug returning a snippet of that variable to confirm,
then coerce with the asText() helper. See references/flow-templates.md → "Gotcha".
7 — Generate a regression eval suite (golden set)
The gate stops bad output at runtime; an eval suite stops a bad change before it ships — so every
agent gets a small golden set that fails loudly if a future edit breaks the gate. Build it from the
two smoke inputs you already have. Read references/evals.md for the exact, verified API contract
— it matters which path you use: the MCP tools as_create_eval_case/as_run_eval need
AGENT_STUDIO_API_KEY and return 401 when it's missing, so fall back to the authenticated app REST
API (POST …/evals, …/evals/{id}/cases, …/evals/{id}/run). Two stable, content-agnostic cases:
- BLOCK — the deliberately-bad input → assertion
{ "type": "contains", "value": "BLOCKED" }.
- PASS — a valid input → assertions
{ "type": "not_contains", "value": "BLOCKED" } + { "type": "json_valid" }.
Create the suite with runOnDeploy: true (re-runs on every deploy). Verify with as_list_evals
(DB-direct, always works): caseCount ≥ 2, lastRunStatus: COMPLETED, ideally lastRunScore: 1.
Not regression-safe until that's green. (Assert on the gate outcome — BLOCKED/not — never on
volatile content, which would fail for the wrong reason.)
8 — Report
Summarize: agent id, model, node/edge count, KB ready, both smoke results (PASS surfaced,
BAD blocked), and the eval suite id + last score. A scaffold isn't "done" until the bad-input case
blocks AND the eval suite is green.
Safe-change discipline (applies to every flow write here)
as_inspect_flow → backup to a file before any change.- Prefer reading the real value and editing it (anchor-replace / exact edit + diff) over retyping
long prompts/code — retyping a 6000-char prompt is how silent corruption sneaks in.
node --check + a tiny functional test on validator code before applying it live.as_patch_node_field for one field; as_update_flow only for structure (full replace, no undo).- Re-inspect after, and smoke the exact case you care about — verify behavior, not just "applied".
Pre-flight checklist (paste into the final report)
References
references/flow-templates.md — exact node/edge JSON, the parametrized validator code (with
input-guard, banned, stat-trace), the emitter + message node templates, and the processor prompt
skeleton. Read it in step 2–3.references/evals.md — the verified eval-suite API contract (MCP vs authenticated REST, the exact
request bodies, all assertion types, and the content-agnostic golden-set design). Read it in step 7.
