agent-behavior-eval-engineer

name	agent-behavior-eval-engineer
description	Design trajectory-level agent security evals for Goal compliance, contract boundaries, goal hacking, unsafe tools, and long-horizon behavior.
version	1.0.0
since	2026-08-29
last_modified	2026-08-29
authors	["platform-engineering"]
stability	stable
min_platform_version	{"codex":"unknown","amazon-q":"unknown","antigravity":"unknown","auggie":"unknown","bob":"unknown","claude-code":"unknown","cline":"unknown","codebuddy":"unknown","continue":"unknown","costrict":"unknown","crush":"unknown","github-copilot":"unknown","gitlab-duo":"unknown","factory":"unknown","forgecode":"unknown","opencode":"unknown","openhands":"unknown","cursor":"unknown","roo-code":"unknown","kiro":"unknown","junie":"unknown","gemini-cli":"unknown","iflow":"unknown","kilocode":"unknown","kimi":"unknown","lingma":"unknown","pi":"unknown","qoder":"unknown","qwen":"unknown","windsurf":"unknown","ollama":"unknown"}
deprecated_since	null
replaces	null
supersedes	[]
changelog	[{"version":"1.0.0","date":"2026-08-29","change":"Initial generated production-ready SDLC / DevSecOps skill"}]

Agent Behavior Eval Engineer

Purpose

Design declarative, reproducible behavioral and adversarial evaluations for complete agent trajectories. Test Goal achievement together with contract compliance, acceptable tool use, state transitions, resistance to goal hacking, and preservation of security boundaries without embedding executable predicates.

Goal and behavioral contract

The authoritative Goal and artifact references are defined in descriptor.yaml. Capability boundaries, identity and delegation requirements, tool permissions, data boundaries, invariants, approval requirements, output contract, and operational limits are defined in contract.yaml. MCP/A2A trust boundaries and the reviewed execution closure live in integrations/ and dependencies.yaml; ASPS and assurance requirements live in assurance.yaml.

Treat those declarations as mandatory execution constraints. skcr validates requirements but does not claim verification or enforce them at runtime.

When to use

• An agentic system needs baseline, adversarial, boundary, or long-horizon security evaluations.
• Goal success can be achieved through forbidden shortcuts, leaked answers, unsafe tools, or specification gaming.
• Tool calls, filesystem changes, network access, secret handling, approvals, and final output must be assessed as one trajectory.
• An incident, threat model, contract, or invariant must become a regression eval.
• Evaluation evidence needs stable scenario, Goal criterion, invariant, capability, and contract-digest references.

Operating model

1. Derive scenarios from Goal criteria, contract boundaries, invariants, threat models, and real failure modes.
2. Define observable initial state, adversarial stimulus, permitted success path, forbidden shortcuts, assertions, and evidence.
3. Evaluate the full trajectory rather than accepting a correct final answer produced through prohibited behavior.
4. Separate deterministic instrumentation from model-based judging and document uncertainty, flakiness, and false-positive controls.
5. Keep eval specifications declarative; place executable harness logic in a separate trusted eval runner.

Spec-Driven Change Context

• Treat repository specs, ADRs, runbooks, change proposals, design notes, and task files as durable context that outlives a chat session.
• For non-trivial changes, prefer a checked-in change artifact or equivalent proposal/design/tasks record before implementation begins.
• Capture requirement deltas explicitly: added, modified, removed, deprecated, or unchanged behavior.