agentic-loop-auditor

스타4

포크0

업데이트2026년 7월 18일 15:25

Audit agent frameworks and agentic workflows for token-burning loop pathologies — unbounded retries, excessive planning depth, tool-call thrashing, context snowballing, missing loop budgets and kill switches. Use this skill whenever the user shares agent configs or traces (LangChain/LangGraph, CrewAI, AutoGen, custom loops), complains an agent is slow or expensive, or is designing any autonomous/multi-step agent. Part of Lean Agentic AI Skills; emits lean-findings.json.

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

출처

navveenb

navveenb/lean-agentic-ai-skills

GitHub 저장소 열기 Creator 저장소 보기

다운로드

Agentic Loop Auditor

Producer skill. Input: agent configs, execution traces/logs, orchestration code. Output: lean-findings.json.

Agents multiply inference: one user request can fan out into dozens of model calls, each carrying a growing context. Loop pathologies are the highest-leverage waste in modern AI systems — and the least audited.

Subject type: emit subject.type: "ai-agent" in findings.

Signatures

Unbounded retries — no max-attempt on failed tool calls or unparseable outputs; the agent argues with the error forever. High. Fix: retry budget (2–3) with backoff, then structured failure surfaced to user.
No step/token budget — loops without max-iterations or cumulative token ceilings. High. Fix: hard budgets with graceful "here's what I have so far" termination.
Context snowballing — full history + full tool outputs re-sent every step; context grows quadratically over the run. High. Evidence from traces: tokens-per-step curve. Fix: tool-output truncation/summarization, sliding window with pinned goal, scratchpad instead of transcript.
Tool-call thrashing — repeated identical or near-identical calls (same search, re-reading the same file) visible in traces. High. Fix: per-run tool memo cache; instruct the model that results persist.
Excessive planning depth — multi-step deliberation, subagent trees, or reflection rounds on tasks a single call solves. Medium-high. Fix: complexity-gated planning — plan only past a triviality threshold.
Max-power model for every hop — the loop's router/formatter steps on the frontier model. Medium; route to model-right-sizer patterns.
Chatty heartbeats/polling — scheduled agent wakeups doing full reasoning to conclude "nothing to do". Medium (a common pathology in always-on personal-agent frameworks). Fix: cheap pre-check (does the checklist have deltas?) before waking the big model; lengthen intervals.
No cost/energy observability — nothing logs tokens per task; waste is invisible. Medium. Fix: per-run receipt (pairs with sci-receipt).

Procedure

If traces exist, compute real numbers: calls per task, tokens per step, duplicate-call rate — these are countable, report them. If only config exists, audit for the absence of guardrails and mark quantitative items out_of_scope.

Honesty rules

Budgets trade against task-completion on genuinely hard tasks — recommend budgets with escalation ("ask the user to continue") rather than silent truncation. Autonomy reductions are product decisions; present them as options with the energy rationale, not mandates.

Cost signal (countable)

Cost drivers: tokens per task (cumulative across loop hops) and tool-call count per task. Both countable from execution traces. Every loop-pathology finding should quote them.

Not this skill's job

Model choice (model-right-sizer), prompt bodies (prompt-carbon-optimizer), when to run the agent (carbon-aware-scheduler).

이 저장소의 다른 Skills

같은 저장소

ai-eval-efficiency-audit

navveenb/lean-agentic-ai-skills

Audit AI evaluation infrastructure for wasted compute — full benchmark suites re-run on unchanged cases, LLM-as-judge grading without caching, oversized judge models, redundant eval passes per commit, and missing result reuse. Use this skill whenever the user shares eval harness configs or CI eval steps, complains that evals are slow or expensive, mentions LLM-as-judge costs, or runs benchmark suites on every change. Part of Lean Agentic AI Skills; emits lean-findings.json.

2026-07-184

api-efficiency-audit

navveenb/lean-agentic-ai-skills

Audit API design for chattiness and over-fetching — REST endpoints requiring N calls per screen, GraphQL resolvers with N+1 fan-out, default-everything payloads, missing pagination and field selection, provider-side polling where webhooks serve, and uncompressed responses. Use this skill whenever the user shares API schemas (OpenAPI, GraphQL SDL), resolver code, endpoint designs, or asks why clients make too many calls or payloads are heavy. Part of Lean Agentic AI Skills; emits lean-findings.json.

2026-07-184

caching-strategy-designer

navveenb/lean-agentic-ai-skills

Design caching layers so work is done once, not per request — HTTP/CDN caching, application-level caches (Redis/Memcached), computed-result caching, and cache-invalidation strategy. Use this skill whenever the user wants to add caching, asks why the same work is recomputed, designs a read-heavy system, or an audit skill (db-efficiency-audit, web-carbon-audit) flags uncached repeated work. Part of Lean Agentic AI Skills; emits a cache design plus lean-findings.json.

2026-07-184

carbon-aware-scheduler

navveenb/lean-agentic-ai-skills

Design time-shifting of flexible workloads (batch jobs, CI, training, backups, crons) to hours and regions when grid electricity is cleanest, using carbon-intensity signals like Electricity Maps, WattTime, or the Carbon Aware SDK. Use this skill whenever the user mentions carbon-aware computing, scheduling flexible/deferrable jobs, demand shifting, running workloads when energy is clean, or asks how to reduce the carbon intensity (not just amount) of their compute. Part of Lean Agentic AI Skills; emits schedule designs plus lean-findings.json.

2026-07-184

carbon-budget-designer

navveenb/lean-agentic-ai-skills

Design per-feature and per-service efficiency budgets — page-weight budgets, token budgets per AI feature, query budgets per endpoint, GPU-hour budgets per training project — with owners, review checkpoints, and enforcement hooks. Use this skill whenever the user wants sustainability targets for teams, asks how to keep efficiency from regressing organizationally, mentions performance-budget-style governance for carbon/energy, or is setting engineering standards. Part of Lean Agentic AI Skills; an advisor emitting budget definitions plus lean-findings.json.

2026-07-184

cloud-waste-audit

navveenb/lean-agentic-ai-skills

Find idle and orphaned cloud resources that burn money and carbon for no output — idle VMs, unattached disks/volumes, unassociated elastic IPs, zombie snapshots, forgotten load balancers, stopped-but-billed instances. Use this skill whenever the user shares cloud billing exports or resource inventories (AWS Cost Explorer/CUR, Azure Cost Management, GCP billing) and the question is about unused or forgotten resources — idle instances, waste, things running that shouldn't be — cost and carbon are two payoffs of the same fix. For costs driven by growing stored data, retention, or tiering (S3/blob bills, Storage Lens reports, "we never delete anything"), use storage-lifecycle-audit instead. Part of Lean Agentic AI Skills; emits lean-findings.json for composition with report/deck skills.

2026-07-184

name

agentic-loop-auditor

description

Agentic Loop Auditor

Producer skill. Input: agent configs, execution traces/logs, orchestration code. Output: lean-findings.json.

Subject type: emit subject.type: "ai-agent" in findings.

Signatures

Unbounded retries — no max-attempt on failed tool calls or unparseable outputs; the agent argues with the error forever. High. Fix: retry budget (2–3) with backoff, then structured failure surfaced to user.
No step/token budget — loops without max-iterations or cumulative token ceilings. High. Fix: hard budgets with graceful "here's what I have so far" termination.
Context snowballing — full history + full tool outputs re-sent every step; context grows quadratically over the run. High. Evidence from traces: tokens-per-step curve. Fix: tool-output truncation/summarization, sliding window with pinned goal, scratchpad instead of transcript.
Tool-call thrashing — repeated identical or near-identical calls (same search, re-reading the same file) visible in traces. High. Fix: per-run tool memo cache; instruct the model that results persist.
Excessive planning depth — multi-step deliberation, subagent trees, or reflection rounds on tasks a single call solves. Medium-high. Fix: complexity-gated planning — plan only past a triviality threshold.
Max-power model for every hop — the loop's router/formatter steps on the frontier model. Medium; route to model-right-sizer patterns.
Chatty heartbeats/polling — scheduled agent wakeups doing full reasoning to conclude "nothing to do". Medium (a common pathology in always-on personal-agent frameworks). Fix: cheap pre-check (does the checklist have deltas?) before waking the big model; lengthen intervals.
No cost/energy observability — nothing logs tokens per task; waste is invisible. Medium. Fix: per-run receipt (pairs with sci-receipt).

Procedure

Honesty rules

Cost signal (countable)

Cost drivers: tokens per task (cumulative across loop hops) and tool-call count per task. Both countable from execution traces. Every loop-pathology finding should quote them.

Not this skill's job

Model choice (model-right-sizer), prompt bodies (prompt-carbon-optimizer), when to run the agent (carbon-aware-scheduler).