원클릭으로 Manus에서 모든 스킬 실행

rumil-find-confusion

스타2

포크1

업데이트2026년 4월 16일 19:04

Scan recent rumil calls for signs of model confusion. Default mode uses fast, free heuristics (error events, non-complete status, exchange errors, cost outliers, thin output). Pass --deep for an LLM-based structured verdict on the top heuristic candidates, cached in scan log. Pass --structural <question_id> for graph health checks on a question's subtree (barren questions, orphans, load-bearing fragile claims). Use when the user wants to review recent runs for quality, triage a batch of calls, or find specific traces worth inspecting.

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

Manus에서 실행

출처

chaosGuppy

chaosGuppy/rumil

GitHub 저장소 열기 Creator 저장소 보기

다운로드

Manus에서 실행

rumil-find-confusion

Triages recent calls in the active workspace, surfacing ones that look off. Three modes:

Heuristic (default) — fast, deterministic, no LLM cost (only local Supabase reads + arithmetic). Scores each recent call by:

hard signals: non-complete status, trace error events, exchange errors
soft signals: cost outliers (> 3× median), thin output relative to input, multiple warnings

Structural (--structural <question_id>) — graph health checks on a question's subtree. Detects barren questions, unjudged questions, orphaned claims, load-bearing fragile claims, dead-end decompositions, and chained supersession. Pure DB queries, no LLM cost.

Deep (--deep) — for the top heuristic candidates, runs a meta LLM call with a shared system prompt (designed for prompt-cache reuse across many scans) and a structured ConfusionVerdict schema. Costs per-scan LLM tokens, roughly:

claude-haiku-4-5-20251001 — ~$0.005 per trace
claude-sonnet-4-6 (default) — ~$0.02-0.05 per trace
claude-opus-4-7 — ~$0.10-0.20 per trace

Exact cost depends on trace size (bigger traces = more tokens in the user message). Verdicts are cached in .claude/state/rumil-scan-log.json, so re-running the skill on a trace already in the log doesn't re-pay. Force a re-scan with --force-rescan.

Override model with --model <id> (e.g. haiku for cheap bulk scans).

Invocation

setopt no_glob 2>/dev/null; set -f; PYTHONPATH=.claude/lib uv run python -m rumil_skills.find_confusion $ARGUMENTS

After it runs

Heuristic output is a ranked list: short ID, call type, signals, score. Deep output adds a structured verdict per call — primary symptom, severity, evidence quotes, suggested action (inspect, redispatch, edit_prompt:<file>, ignore).

For any call that looks worth investigating, the natural next step is /rumil-trace <call_id> to read the full exchanges, or /rumil-review <question_id> if the issue seems question-wide.

이 저장소의 다른 Skills

같은 저장소

rumil-versus-iterate

chaosGuppy/rumil

Iterate on versus's pipeline — review recent (or freshly-fired) completion / judging runs, identify concrete improvement opportunities, test them via /rumil-forks, and consolidate findings into a ranked punch list. Spawns parallel agents for trace investigation and fork experiments so wall-clock stays low. Use when the user wants to "review the recent versus runs and find improvements," "play with these traces," "see what we could do better in d&e / two_phase," or anytime after a versus generation/judging batch when it's worth turning the runs into actionable code/prompt changes. Default scope is one forethought essay × Sonnet × budget 4, but every input is overridable.

2026-05-042

rumil-system

chaosGuppy/rumil

Background knowledge for working with the rumil research workspace from Claude Code. Explains the two-lane provenance model (rumil-mediated vs cc-mediated), when to use which rumil-* skill, how workspace/session state works, and the visibility/attribution story. Auto-load this whenever the user asks about rumil, wants to inspect research, dispatch a call, discuss a question, apply moves, or review a trace.

2026-05-022

rumil-versus-complete

chaosGuppy/rumil

Run orchestrator-driven essay completions for the versus pairwise-eval pipeline. Per essay × prefix × model × workflow, fires a rumil orchestrator (TwoPhase, DraftAndEdit) and writes the resulting continuation as a `versus_texts` row tagged `kind="completion"`, `source_id="orch:<workflow>:<model>:c<hash8>"` — pickable as a contestant by `rumil-versus-judge` afterwards. Use when the user wants to A/B orchestrator-produced continuations against single-shot model continuations or against the human baseline. Both `--orch two_phase` and `--orch draft_and_edit` are usable. For single-shot completions (no orch), use `rumil-versus-generate`.

2026-05-022

rumil-versus-judge

chaosGuppy/rumil

Run pairwise judgments on versus essay-continuation pairs. Default mode is the unified blind path (single-turn LLM call, no tools, no DB) — claude-* models route direct to Anthropic, others through OpenRouter. --variant orch fires a full TwoPhaseOrchestrator run per pair. Use when the user wants to measure how rumil discriminates on pairs with known ground truth (human continuation vs. model continuations), compare blind judges against workspace-aware ones, or top up pending judgments after adding new dimensions or models.

2026-05-022

rumil-load-run

chaosGuppy/rumil

Load a rumil run into the conversation by run ID. Shows a tree of every call in the run (id, type, status, cost, event summary) grouped under the run. Use when you have a trace URL like /traces/<run_id> and want to see the whole run, not a single call. For one call's verbatim trace use rumil-trace instead. Pass --compare <other_run_id> to A/B two runs side-by-side.

2026-05-022

rumil-trace

chaosGuppy/rumil

Dump a rumil call's full execution trace — trace events plus every LLM exchange verbatim (system prompt, user message, response, tool calls). Use to inspect what happened inside a specific call, debug model confusion, or review a run for quality. Takes a short (8-char) or full call ID.

2026-05-022

name	rumil-find-confusion
description	Scan recent rumil calls for signs of model confusion. Default mode uses fast, free heuristics (error events, non-complete status, exchange errors, cost outliers, thin output). Pass --deep for an LLM-based structured verdict on the top heuristic candidates, cached in scan log. Pass --structural <question_id> for graph health checks on a question's subtree (barren questions, orphans, load-bearing fragile claims). Use when the user wants to review recent runs for quality, triage a batch of calls, or find specific traces worth inspecting.
allowed-tools	Bash
argument-hint	[--limit N] [--deep [--deep-limit K] [--model ...]] [--force-rescan] [--structural <question_id>]

rumil-find-confusion

Triages recent calls in the active workspace, surfacing ones that look off. Three modes:

Heuristic (default) — fast, deterministic, no LLM cost (only local Supabase reads + arithmetic). Scores each recent call by:

hard signals: non-complete status, trace error events, exchange errors
soft signals: cost outliers (> 3× median), thin output relative to input, multiple warnings

claude-haiku-4-5-20251001 — ~$0.005 per trace
claude-sonnet-4-6 (default) — ~$0.02-0.05 per trace
claude-opus-4-7 — ~$0.10-0.20 per trace

Override model with --model <id> (e.g. haiku for cheap bulk scans).

Invocation

setopt no_glob 2>/dev/null; set -f; PYTHONPATH=.claude/lib uv run python -m rumil_skills.find_confusion $ARGUMENTS

After it runs

For any call that looks worth investigating, the natural next step is /rumil-trace <call_id> to read the full exchanges, or /rumil-review <question_id> if the issue seems question-wide.