Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

mech-interp-runner

Estrellas0

Forks0

Actualizado10 de julio de 2026 a las 16:27

Run, plan, validate, or aggregate Epistemic-Humility local mechanistic-interpretability sweeps, including hidden-state candidate inventories, causal-pilot sweep planning, explicit non-GPU/GPU gates, base-original skip handling, and offline result aggregation. Use when working on local mech-interp sweeps, causal-pilot diagnostics, activation-addition/logit-diagnostic runs, behavior-axis scans, SAE feature screens, or future reruns of the legacy full candidate inventory.

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

ProfSynapse

ProfSynapse/Epistemic-Humility-Research

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Científicos de datosOcupaciones informáticas y matemáticas·SOC 15-2051

Explorador de archivos

15 archivos

SKILL.md

readonly

Más de este repositorio

mismo repositorio

mechinterp-cells

ProfSynapse/Epistemic-Humility-Research

Author, organize, and launch tuner-backed mechanistic-interpretation cells (activation reading and writing) for the Epistemic-Humility project. Use when designing a new steering / extraction / probe-fit / gate-scoring cell, deciding where its config, direction, grader, and outputs live, or launching one locally or on Modal. This skill is about USING the synaptic-tuner `mechinterp` verbs via declarative recipe YAML plus project plug-ins; it never modifies the tuner submodule and never touches the frozen legacy machinery.

2026-07-200

experiment-runner

ProfSynapse/Epistemic-Humility-Research

Operational runbook for the Epistemic-Humility Locked Matrix Experiment Runner - expands the PROTOCOL v0.3 (LOCKED) run matrix (3-seed headline + LR/beta sensitivity panel at 4B, 3-seed confirm at 8B, 2 bridge replication cells) into per-cell tuner invocations across two lanes (local RTX 3090 / HF Jobs cloud), with hard pre-registration count assertions, prerequisite gating, data staging, and committed provenance run records. Use when launching, dry-running, or gating the locked matrix, materializing per-cell recipes, or inspecting run records. This skill is about USING the runner via checked-in scripts; it never modifies the synaptic-tuner submodule.

2026-07-180

experiments

ProfSynapse/Epistemic-Humility-Research

Lifecycle tooling for the experiments-first repo layout - one self-contained directory per evidence-producing experiment (steer cell, training run, eval, probe-fit, or lab diagnostic) under experiments/, each with a signed AMENDMENT.md, a thin machine-readable experiment.yaml manifest, pinned instrument configs, and generated indices. Use to scaffold, sign, list, show, resolve, and validate experiments, to regenerate the registry, and to understand the draft -> sign -> run -> resolve lifecycle and the shared-input promotion rule. The bin/exp wrapper runs these scripts; validation and registry regeneration run at commit time via .githooks/pre-commit.

2026-07-170

data-exhaust

ProfSynapse/Epistemic-Humility-Research

Package a terminal experiment's data exhaust (aggregate stats and, where license-permitted, row-level generation text) into reproducible Hugging Face datasets, with a license gate, containment lint, and a thin uploader. Use when an experiment under experiments/<slug>/ reaches a terminal verdict (resolved, null-result, or falsified) and its analysis-committed/ artifacts should be published.

2026-07-140

family-atlas

ProfSynapse/Epistemic-Humility-Research

Standardize the per-model layer-mapping extraction (full-depth hidden-state capture, workspace-dimension profile, three-axis held-out read panel) so any new model, family, or size entering the Epistemic-Humility program gets a fast, cheap, uniform layer atlas before any actuation design. Use when a new model/family/size enters the program, before designing any per-family actuation cell, or when an existing family's substrate changes (new revision, new checkpoint).

2026-07-120

knowledge-graph

ProfSynapse/Epistemic-Humility-Research

Create, validate, normalize, export, and analyze typed Obsidian knowledge graph metadata in the Professor Synapse vault, including `kg` node metadata, `relationships` edge objects, `related` link projections, ontology-governed edge types, legacy relationship shorthand, and graph analysis reports.

2026-07-100

name	mech-interp-runner
description	Run, plan, validate, or aggregate Epistemic-Humility local mechanistic-interpretability sweeps, including hidden-state candidate inventories, causal-pilot sweep planning, explicit non-GPU/GPU gates, base-original skip handling, and offline result aggregation. Use when working on local mech-interp sweeps, causal-pilot diagnostics, activation-addition/logit-diagnostic runs, behavior-axis scans, SAE feature screens, or future reruns of the legacy full candidate inventory.
allowed-tools	Read, Bash, Write, Grep, Glob

Mech-Interp Runner

This skill drives the frozen bespoke local mech-interp machinery (see archive/experiment/phase1/probe/steering/LEGACY.md). It mints NO new cells. To author a NEW steering / extraction / probe-fit / gate-scoring cell, use the mechinterp-cells skill (tuner-backed mechinterp verbs), not this one.

Use the checked-in scripts and configs to plan, gate, run, and aggregate local mech-interp work. Do not hand-roll terminal loops. This SKILL.md is a progressive-disclosure router: it carries the invariants and routing only. Procedural detail lives in references/, and current results live in docs/sessions/ and experiment-local docs under experiments/<slug>/ - never inline run history here.

Start Here

Always choose the narrowest reference needed for the task:

Task	Load
Decide whether a candidate is a real result; interpret directions	references/interpretation-invariants.md
Preflight or configure a hidden-state extraction (incl. attention-head granularity)	references/extraction-preflight.md
Plan / execute / aggregate / observe a causal-pilot sweep; runtime-identity semantics	references/sweep-workflow.md
Launch a long GPU sweep that must survive CLI teardown (checkpoint/resume)	references/resumable-gpu-sweeps.md
Choose rows / fixed row keys / rare-cell panels / per-row logit targets	references/row-selection.md
Run next-token logit diagnostics and required control arms	references/logit-diagnostics.md
Run the generated-answer replay behavioral gate	references/generated-replay.md
Screen SAE features	references/sae-path.md
Run behavior-axis scans, plane/multicell readouts, direction transforms, multi-layer candidates	references/behavior-axis-path.md
Test answer-sycophancy / helpfulness pressure across regimens	references/sycophancy-probe-path.md
Test whether a read-side finding generalizes to a second known/unknown dataset	references/cross-dataset-transfer.md
Test WHEN an axis fires vs the decision token (pre-commitment vs decision-echo, read-only)	references/read-trajectory-timing.md

Do not preload all references. Read only the file needed for the current operation, then follow any further routing inside it.

Quick Commands

Prefer the skill CLI for common local analyses instead of hand-writing long PowerShell commands. It delegates to checked-in repo scripts, resolves paths from the repo root, uses sys.executable, and forces UTF-8 subprocess output (PYTHONUTF8=1, PYTHONIOENCODING=utf-8, PYTHONUNBUFFERED=1) to avoid Windows console encoding failures.

Task	Command
Discover subcommands	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py --help`
Quick non-GPU validation	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py validate --quick`
Behavior-axis scan	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py behavior-axis-scan --config <cfg>`
Causal sweep (plan/materialize)	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py causal-sweep --config <cfg> --mode-filter logit_diagnostic --write-plan --materialize-configs`
Sycophancy generation analysis	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py sycophancy-generation-analysis --generations <jsonl> --output-root <dir>`
Cross-dataset panel (step 1)	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py xdataset-build-panel --source <jsonl> --dataset <id> --out-dir <dir> --n-known 600 --n-unknown 400`
Cross-dataset behavior rows (step 3)	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py xdataset-behavior --generation <rows.jsonl> --panel-rows <gen_rows.jsonl> --out-dir <dir>`
Residual caution direction (read-trajectory step 1)	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py residual-caution-direction --extraction-dir <ext> --behavior-rows <rows.jsonl> --layer 35 --out <dir>/caution_direction_L35.json`
Read-trajectory re-analysis (GPU-free)	`python .skills/mech-interp-runner/scripts/mechinterp_cli.py residual-read-trajectory-analysis --rows <rows.jsonl> --out <analysis.json>`

Use --dry-run on any subcommand to print the delegated command before running it. For live Docker/GPU execution the same approval rule applies: do not pass --execute unless the user has approved that live run.

Core Invariants

The research target is coherent epistemic-humility expression, not a raw refusal axis. A candidate is promising only if it improves one damaged behavior without degrading the paired desired behavior. See references/interpretation-invariants.md.
Generated-answer replay is the behavioral gate. Offline separability (AUC, d, readout recall) and first-token answer-start movement are screening evidence, never a steering claim.
Treat outputs as Tier 2 exploratory local mechanism evidence.
Do not edit synaptic-tuner/.
Do not run Docker/GPU unless the user explicitly approves that live run.
Keep base-original h_base adapterless work fail-closed until the live runner explicitly supports adapterless base execution. In DPO/KTO extractions h_base is the SFT-merged pre-adapter model, not original Qwen base.
Keep generated outputs gitignored by default unless a governed publication decision explicitly whitelists them.
Stay local to this repository for mech-interp work. Do not use external workflow or memory systems unless the user explicitly asks in the current turn.

Validation

Run focused non-GPU checks after runner/config/skill edits:

python -m pytest experiments/common/knowledge_probe/tests/test_mechinterp_causal_pilot_sweep.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_causal_pilot_runner.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_causal_pilot_dry_run.py -q
python -m pytest experiments/common/knowledge_probe/tests/test_mechinterp_behavior_axis_scan.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_behavior_axis_directions.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_calibrated_expression_plane.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_logit_cell_analysis.py -q
python -m pytest experiments/common/knowledge_probe/tests/test_mechinterp_sae_smoke.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_sae_train.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_sae_feature_analysis.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_sae_feature_directions.py \
  experiments/common/knowledge_probe/tests/test_mechinterp_sae_behavior_feature_analysis.py -q
python -m pytest .skills/mech-interp-runner/tests/test_mechinterp_cli.py -q
python bin/sync_skills.py --check

Compact Recovery Loop

After context compaction or degraded memory:

Re-read this skill; drill into references/ only as needed.
Re-read the latest relevant docs/sessions/ note and local KG/search results if they affect the next experiment.
Set up or run the current local experiment with checked-in scripts/configs.
Analyze outputs against the research target (interpretation-invariants).
Update session notes, experiment-local runbooks/plans, and this skill when a durable procedure or gotcha changes.
Choose the next highest-ROI local slice from the evidence and repeat.

Skill Maintenance

Edit the canonical tree under .skills/mech-interp-runner/ only; .agents/ and .claude/ are generated mirrors. Keep this contract intact:

Put timeless rules, routing, commands, and validation gates in SKILL.md.
Put durable procedure and gotchas in one-level references/*.md files and link them from Start Here.
Put current findings, numeric results, and interpretation snapshots in docs/sessions/*.md and experiment-local docs under experiments/<slug>/ - not in the skill.
Put reusable analysis logic in checked-in scripts/configs, not prose.
If a section grows because of one experiment, move the details to a session or experiment note and leave only the general rule plus the reference path.

Before adding a paragraph here, ask: "Will this still guide a future agent six months from now without loading today's run history?" If not, it belongs in a reference or a note. After canonical edits, run:

python bin/sync_skills.py --write --skill mech-interp-runner
python bin/sync_skills.py --check --skill mech-interp-runner