| name | ask |
| description | Prompt any model or alias via "ask <model> <question>". Resolves short names (deepseek, kimi, qwen, glm) to full model IDs. Captures session IDs for follow-up questions. Comparison mode: "ask deepseek kimi <question>" dispatches multiple models in parallel. Each call is a full Hermes agent with tools and multi-turn reasoning. Replies inline with a model badge.
|
| version | 1.2.0 |
| author | agent |
| license | Apache-2.0 |
| platforms | ["linux","macos"] |
| metadata | {"hermes":{"tags":["multi-model","prompt","productivity","alias"],"related_skills":["advisors","dev","next-best-questions","investigator"],"config":[{"key":"ask.enabled","description":"Enable the ask skill","default":true,"prompt":"Enable ask skill?"}],"category":"productivity"}} |
Ask — Prompt Any Model by Alias
Overview
When the user says "ask ", dispatch that
model via hermes chat -q and reply inline with the response. Each call is a
full Hermes agent — tools, skills, multi-turn reasoning. Short aliases resolve
to full model names so the user doesn't need to remember exact IDs.
Beyond the chat command, this skill is also the model-dispatch library for other skills:
scripts/model_utils.py (alias resolution, prompt building, dispatch_single) is imported at
runtime by next-best-questions and investigator, resolved via
$HERMES_HOME/skills/productivity/ask/scripts — which is why hub installs must place this skill
under the productivity category, and install it first.
"ask deepseek What is ACID compliance?"
→ 🤖 deepseek-v4-pro:cloud (7.2s)
────────────────────────────────────────
ACID compliance ensures that database transactions are...
"ask deepseek and kimi Should we use PostgreSQL or MongoDB?"
→ Side-by-side responses from both models
When to Intercept
Intercept when the user's message starts with "ask" followed by a known
model alias, full model name, or a fuzzy match:
- "ask deepseek ..." → dispatch deepseek-v4-pro:cloud
- "ask kimi ..." → dispatch kimi-k2.7-code:cloud
- "ask qwen ..." → dispatch qwen3.6:35b-a3b (changed from qwen3-coder-next:q4_K_M)
- "ask deepseek kimi ..." → comparison mode (both models in parallel)
- "ask deepseek-v4-pro:cloud ..." → full model name also works
- "ask minimax-3 ..." → fuzzy match → dispatch minimax-m3:cloud (~0.5s LLM lookup)
- "ask ds-pro ..." → fuzzy match → dispatch deepseek-v4-pro:cloud
Do NOT intercept if "ask" is not the first word, or if no model alias
follows it. "Can I ask you something?" is a normal question.
Fuzzy Alias Resolution
Two-tier model name resolution:
- Exact match (instant, free) — checks the
ALIASES dict directly
- LLM fuzzy fallback (~0.5s) — if no exact match, asks the fast local
model (
qwen3.6:35b-a3b via raw Ollama API) to map the input to the
closest known alias
This means ask minimax-3, ask ds-pro, ask qw all work even though
they're not exact aliases. The LLM matches by brand, family, version, or
abbreviation. Results are cached so repeated fuzzy lookups are instant.
Failures are graceful: if Ollama is down or the LLM can't match, the
original name is passed through unchanged (Hermes will reject it with a
clear "unknown model" error).
Alias Registry
| Alias | Model | Notes |
|---|
deepseek, deepseek-pro, ds, dsp | deepseek-v4-pro:cloud | Heaviest reasoning |
deepseek-flash | deepseek-v4-flash:cloud | Faster, lighter |
kimi, kimi-k2, kimi-coder | kimi-k2.7-code:cloud | Code analysis, debugging |
qwen, fast, local, qwen-local | qwen3.6:35b-a3b | Local standard — 35B MoE, 114 tok/s, 4.4s wall |
qwen-coder | qwen3-coder-next:q4_K_M | Code generation, fast local (unchanged) |
gemma | gemma4:12b-mlx-bf16 | Lightweight local fallback |
glm, glm-5 | glm-5.2:cloud | Broad reasoning, planning |
phi, phi-reasoning | phi4-reasoning:plus | Reasoning |
minimax, minimax-m3, mm, mm3 | minimax-m3:cloud | General reasoning, MiniMax M3 via Ollama proxy |
devstral | devstral-small-2:24b-cloud | Code |
gpt-oss | gpt-oss:120b | Open-source GPT |
llama | llama4:scout | Meta's latest |
planner | glm-5.2:cloud | Dev role alias |
coder, qa-tester, qa, tech-docs, docs | qwen3-coder-next:q4_K_M | Dev role aliases |
debugger | qwen3-coder-next:q4_K_M | Primary debugger — fast local, cascades to fallback on failure |
debugger-fallback | kimi-k2.7-code:cloud | Fallback debugger — cloud, stronger reasoning |
test-planner | deepseek-v4-pro:cloud | Test suite design — multi-suite (unit/integration/e2e) |
Full model names (e.g., deepseek-v4-pro:cloud) also work — pass through as-is.
How to Dispatch
Step 1 — Parse the user's message
Extract the model alias(es) and the question:
"ask deepseek What is the best way to handle connection pooling in Python?"
└─ alias ──┘ └───────── question ─────────────────────────────────┘
"ask deepseek kimi Should we use SQLite or PostgreSQL?"
└─ aliases ─┘ └──── question ────┘ → comparison mode
Important: If the prompt contains -- or flag-like text (e.g., "ask deepseek
What does --max-turns mean?"), use the --prompt flag to avoid argparse
confusion:
python3 ask.py deepseek --prompt "What does --max-turns mean?"
python3 ask.py --models deepseek,kimi --prompt "Should we use --verbose or --quiet?"
Step 2 — Run the script
python3 /opt/data/skills/productivity/ask/scripts/ask.py \
deepseek "What is ACID compliance?"
python3 /opt/data/skills/productivity/ask/scripts/ask.py \
deepseek --prompt "What does --max-turns mean?"
python3 ask.py --models deepseek,kimi --prompt "Should we use PostgreSQL or MongoDB?"
Or via execute_code for structured handling:
import subprocess
result = subprocess.run([
"python3", "/opt/data/skills/productivity/ask/scripts/ask.py",
"deepseek", "What is ACID compliance?",
], capture_output=True, text=True, timeout=3600)
print(result.stdout)
Step 3 — Reply to the user
Present the model's response inline with a badge:
🤖 **deepseek-v4-pro:cloud** (7.2s)
ACID compliance ensures that database transactions are Atomic, Consistent,
Isolated, and Durable...
For comparison mode, present side-by-side:
🤖 **deepseek-v4-pro:cloud** (7.2s)
PostgreSQL — ACID is native...
🤖 **kimi-k2.7-code:cloud** (5.9s)
PostgreSQL — battle-tested transactions...
📊 2/2 models responded
Thinking Levels (Reasoning Effort)
Control how deeply the model reasons before responding. This maps directly to
Hermes' agent.reasoning_effort config setting, which controls extended thinking
tokens (chain-of-thought) for models that support it.
python3 ask.py fast "What is 2+2?" --thinking none
python3 ask.py deepseek "Design a REST API" --thinking medium
python3 ask.py deepseek "Prove this algorithm is O(n log n)" --thinking xhigh
| Level | Description | When to use |
|---|
none | Disable reasoning entirely | Simple facts, quick queries, "what is X" |
minimal | Bare minimum reasoning | Quick responses with slight deliberation |
low | Light reasoning | Most quick questions, good balance |
medium | Moderate reasoning | General-purpose tasks, default for most work |
high | Deep reasoning | Complex analysis, code review, architecture decisions |
xhigh | Maximum reasoning | Very hard problems, proofs, deep analysis |
How it works: The --thinking flag sets agent.reasoning_effort via
hermes config set before the hermes chat -q call, then restores the
original value in a finally block. This means:
- The change is temporary — only applies to this one call
- If the call crashes or times out, the original value is still restored
- Comparison mode applies the same thinking level to all models in the batch
- If
--thinking is omitted, the current config value is used (no change)
Comparison mode serialization: When --thinking is used with multiple
models, dispatch runs sequentially (not in parallel) to avoid a race
condition on the global agent.reasoning_effort config. Without --thinking,
comparison mode runs in full parallel as before. A warning is printed to stderr
when serialization is active.
TODO: Once hermes chat supports --reasoning-effort as a per-call CLI
flag, the serialization workaround can be removed and comparison mode will
run in full parallel even with thinking levels. Verified absent 2026-07-11,
blocked upstream — the serialization workaround documented here must stay.
Not all models support reasoning. Cloud models (DeepSeek, GLM) and some
local models (Qwen3, Gemma4) support it. The reasoning effort is silently
ignored for models that don't. The model's response quality will reflect the
thinking level when supported.
Output metadata includes the thinking level when using -o:
Comparison Mode
When the user specifies multiple models, dispatch them in parallel:
python3 ask.py deepseek kimi qwen "Should we use PostgreSQL or MongoDB?"
python3 ask.py deepseek kimi "Review this code" --context "$(cat auth.py)"
The script runs all models in parallel via concurrent.futures and prints
results with badges. Report them to the user as they arrive.
Session Memory
The script captures session_id from each hermes chat call and saves it to
~/.hermes/ask-sessions.json. This enables follow-up questions:
python3 ask.py deepseek "Design a REST API for task management"
python3 ask.py deepseek "Now add WebSocket support for real-time updates"
python3 ask.py deepseek "Elaborate on the auth strategy" --resume 20260627_120000_abc123
python3 ask.py --sessions
python3 ask.py --clean-sessions
For the controller: When the user says "ask deepseek to elaborate" or
"ask deepseek follow up on that", use --resume with the saved session ID.
The script auto-resumes if a session exists for the alias.
Raw Mode (Direct Ollama API)
When you need a fast model response without the full Hermes agent loop
(tools, skills, system prompt), use --mode raw:
python3 ask.py fast "What is 2+2?" --mode raw
| Mode | Speed | Use case |
|---|
agent (default) | 7-52s | Full Hermes agent with tools, skills, multi-turn reasoning |
raw | 0.5-3s | Direct Ollama API call — classification, simple Q&A, fast lookups |
Raw mode calls the Ollama API directly (like triage.py) instead of going
through hermes chat -q. It's ideal for fast classification, simple
questions, and any case where you don't need the agent loop overhead.
File Output
For pipeline use (feeding output into other tools), use -o:
python3 ask.py deepseek "Plan the architecture" -o /tmp/plan.md
Files include a metadata header:
Arguments
| Flag | Description |
|---|
<alias(es)> <prompt> | Positional: model alias(es) then the prompt (no -- or special chars) |
-p, --prompt | Prompt text (use when prompt contains -- or special chars) |
--models | Comma-separated model aliases (alternative to positional) |
--context | Context string to include |
-c, --context-file | Read context from file |
-o, --output | Output file (single model only, includes metadata header) |
-t, --toolsets | Toolsets (default: file,web) |
--max-turns | Max agent turns (default: Hermes config agent.max_turns) |
--timeout | Timeout seconds (default: 3600) |
--thinking | Reasoning effort: none, minimal, low, medium, high, xhigh |
--mode | Dispatch mode: agent (full Hermes agent, default) or raw (direct Ollama API, ~0.5s) |
--resume | Resume a previous session by ID |
--sessions | List all saved sessions |
--session <alias> | Show session info for an alias |
--clean-sessions | Remove expired sessions (TTL: 1 hour) |
--cwd | Working directory for the dispatched agent (single model only; warns in comparison mode) |
--emit-events | Print JSON dispatch events to stderr for programmatic callers (e.g., 2>events.jsonl) |
Invoking ask like an agent
Use --emit-events when another program is supervising the call. Both
ask.py and pipeline.py write one flushed JSON object per line to stderr;
their result remains on stdout (the pipeline's full result is available with
--json). The exit-code contract is: 0 success, 1 failure, 2
needs-human, and 10 raw suspended workflow. Exit 10 belongs to the
resumable-script runtime; gate_driver.py converts an unanswered gate to 2.
The current event catalog is:
| Event | Fields in addition to event |
|---|
dispatch_start | model, role, thinking, timestamp |
dispatch_end | model, elapsed, success, chars, error |
fallback | model, notice |
triage_done | category, confidence |
routing_decision | skill, model, thinking, toolsets |
dispatch_retry | attempt, reason |
devloop_start | pipeline_mode |
devloop_end | pipeline_mode, terminal |
auto_answer | question, answer, round, seam |
dispatch_single() removes Hermes fallback notices from content with
clean_output_full() and returns the first such notice as fallback (also
emitted as the fallback event). This matters for model comparisons: hermes chat can silently reroute a nonexistent or unavailable model tag and still
answer with exit 0. An ask <bad-model-tag> invocation can therefore appear
to work while a different model answered. Check fallback, not only the exit
code.
The dispatch defaults are single-sourced in model_utils.py:
DEFAULT_PROVIDER, DEFAULT_TIMEOUT (3600 seconds), DEFAULT_TOOLSETS, and
DEFAULT_MAX_TURNS. timeout applies to each dispatch_single() subprocess
attempt; pipeline retries can therefore consume up to (max_retries + 1) × timeout. Empty output records Empty output (exit N) and a returncode; among
empty outputs, only exit 0 is retryable. API, rate-limit, connection, and
timeout errors are also transient retry candidates.
--auto-answer is optional on both ask.py and pipeline.py. With no answer
model argument, ask uses the selected model and pipeline uses the routed model.
It recognizes free-text clarifying questions, generates at most two answers,
and resumes the same session. Both retain an auto_answers audit list; an
unanswered pipeline question is returned as needs_human, exit 2, with a
pending_question. The underlying answer helper gives an answer artifact
precedence over dispatcher stdout when an artifact-capable caller provides one;
the durable gate driver does so with its state directory.
Durable resumable-script gates use the related gate_driver.py seam. Its
sequence is run → raw exit 10 → inspect pending gate → answer an enum gate
(case-insensitively normalized to the declared option, with one off-menu retry)
→ resume. It returns 0 when completed, 2 for an unanswered/capped gate, and
1 for errors. For example:
python3 scripts/gate_driver.py --flow /tmp/flow.yaml --state-dir /tmp/flow-state \
--input '{"request":"ship it"}' --auto-answer --json --emit-events
Without --auto-answer, the driver performs the run/inspect portion and
surfaces the pending gate as exit 2; with it, the driver resumes using the
enum-normalized answer and records an auto_answer event with seam: "gate".
Pitfalls
Design Principle: Ask Is a Dumb Pipe — No Control Channel
The ask skill is a dispatch primitive, not an orchestrator. It takes a model
alias + prompt, runs hermes chat -q, and returns the output. That's it.
Do NOT add:
- New orchestration control channels (file-based handoff or unbounded event/state machinery)
- State machines or iteration loops
- Phase-to-phase data passing via filesystem
- Session continuity beyond
--resume and the bounded auto-answer continuation
Why: A 3-seat advisor panel (DeepSeek + Kimi + Qwen, GLM synthesis) reviewed
this question on 2026-06-28 and unanimously concluded: the control channel
pattern belongs in orchestrators (SDLC pipeline, Kanban workers), not in
dispatch primitives (ask, prompt_model). Adding control-channel complexity
to ask would:
- Duplicate infrastructure — SDLC already has
sdlc_state.py,
sdlc_worktree.py, and the v6 state machine. Ask would need its own copies.
- Blur the primitive/orchestrator boundary — ask's value is simplicity.
Making it stateful undermines its role as a building block.
- Create maintenance burden — every control-channel feature added to ask
must be kept in sync with the SDLC implementation.
The right pattern: Keep ask's supported invocation seams small
(--emit-events, --resume, and bounded --auto-answer). When you need
file-based handoff, multi-phase iteration, or richer orchestration, use the
pipeline/devloop route or build an orchestrator that composes ask as a
sub-component.
This was the core finding of the 2026-06-28 control-channel review thread
(planner → Kimi → Qwen → 3-seat advisor panel → 8 improvement proposals
implemented). The ask skill emerged cleaner (consolidated dispatch_comparison,
deleted sdlc_control.py, added --cwd and --emit-events as simple CLI
flags), while the SDLC orchestrator absorbed the control-channel complexity
where it belongs.
Historical (sdlc.py era, retired 2026-07-01): SDLC Pipeline: All child processes get all tools (2026-06-29 policy)
Per user policy, every dispatched child process in the SDLC pipeline gets full tool access (toolsets='file,terminal,web'). This reverses the earlier toolsets='' policy for text-output phases. The rationale: models should have the tools they need to do their job, and restricting tools caused more problems (empty output, inability to read files for context) than it solved. All v5 and v6 dispatch sites now use toolsets='file,terminal,web': implement, tech_docs, simplify_code, council_review, debug_cascade, plan, design_test_suites, and all v6 state machine phases. The max_turns parameter still defers to Hermes config (max_turns=None).
Historical (sdlc.py era, retired 2026-07-01): SDLC Pipeline: extract_python_code() leniency → ast.parse() verification
extract_python_code() in sdlc.py went through two rounds of fixes:
Round 1 (P14-A): Strategy 2 (unlabeled code blocks) was too strict — only
returned code containing def, import, class, or print(. Fixed by adding
return and if __ to the keyword list, plus a lenient fallback. Strategy 3
threshold lowered. Also changed Strategy 1 from re.search (first match) to
re.findall + max(blocks, key=len) (largest block).
Round 2 (P14-A-2, advisor A1): The lenient fallback was dangerous — it could
return prose, API error text, or markdown as "code." Now ALL three strategies
verify syntax with ast.parse() before returning. If a block doesn't parse as
valid Python, it's rejected. This prevents the pipeline from executing prose as
code. Edge cases covered by 9 tests in TestExtractPythonCode (P13-H):
multiple blocks → largest valid, prose → None, API errors → None, invalid
syntax → None, mixed valid/invalid → returns valid one.
Historical (sdlc.py era, retired 2026-07-01): SDLC Pipeline: implement_failed guard
When extract_python_code() returned None, the pipeline reported
pipeline_status='success' because run_verification and extracted_code was
falsy, skipping tests entirely. Fixed by adding an explicit guard: if
extracted_code is None after the implement phase, return
pipeline_status='implement_failed' with error 'no extractable code from implement phase'.
Historical (sdlc.py era, retired 2026-07-01): SDLC Pipeline: extract_python_code() takes first block, not largest
Strategy 1 used re.search (first match only). Models may emit multiple
`re.findall` + `max(blocks, key=len)` to return the largest block.
`simplify_code()` runs after tests pass, but simplified code is **never
re-tested**. If simplification breaks something, the pipeline reports success
with broken code. P12-B confirmed this: simplify produced 1487 chars of code
that was never verified. Fix (P14-D): re-execute test suites against simplified
code; revert to pre-simplification code on failure. Add
`re_verify_after_simplify` param (default True).
`council_review()` dispatches to 3 models in parallel. If one fails silently,
the council result is incomplete but not flagged. Fix (P14-E): track per-seat
success/failure; mark result as 'partial' if some seats fail; include 'seats'
list with per-model status. Pipeline should not fail on partial council
(advisory only).
Full 9-phase pipeline takes 324s (5.4 min) for a simple palindrome checker
(P12-B). `run_pipeline()` timeout=3600s. Fix (P14-F): accept
separate `pipeline_timeout` param (default 900s for SDLC mode); per-phase
timeout stays at 120s.
This concern is resolved more strongly than the earlier proposed
`validate_tests()` phase. Devloop's DoD oracle first runs
`check_structural_coverage()` so every criterion owns a test, then uses
`judge_assertions()` to have two independent judge models — neither the
implementer — verify that each criterion's test set encodes it. A disagreement
fails closed to `HUMAN_REVIEW` unless its optional tiebreaker resolves the split.
The admitted tests are frozen while the coder works; the oracle may be replaced
only through one bounded, re-admitted redesign cycle.
Build and debug requests on the live, enabled `devloop_bridge` route inherit
this oracle automatically.
`debug_cascade()` passes `error_feedback` to attempt 2 (kimi), but doesn't
include the original failed code. Kimi needs both to fix it. Fix (P14-G):
include original code + error feedback in attempt 2 prompt with instruction
"Fix the code above based on this error: ...".
### Historical (sdlc.py era, retired 2026-07-01): SDLC Pipeline: AI-generated tests need linting too
`design_test_suites()` generates test code that the pipeline runs against the
generated implementation. But AI-generated tests can have the same style issues
as AI-generated code — trailing whitespace, mixed tabs/spaces, unused imports.
Added `lint_test_suites()` (Phase 3.6) that lints each test suite individually
with the same aggressive auto-fix pipeline used for code (ruff check --fix
--unsafe-fixes → ruff format → autopep8 --aggressive --aggressive → ast.parse
re-verify). Returns `fixed_suites` dict with auto-fixed test code. Test syntax
errors produce a warning but don't block the pipeline — pytest will fail and
the debug cascade handles it. Discovered Jun 2026 during P14-H implementation.
`tech_docs()` (and other text-output phases like `simplify_code()`) can return
`None` content on some runs even when the same phase succeeded on a prior run.
P12-C passed on first run (327s) but the second run (432s) failed because
tech_docs pass 1 returned None. This is model non-determinism — the same prompt
with the same model sometimes produces empty output. Mitigation: enhancement
phases (docs, simplify, council) should be treated as optional — the pipeline
should log a warning and continue rather than crashing. Tests should check for
None and skip assertions gracefully rather than hard-failing. Discovered
Jun 2026 during P12 live E2E re-runs.
When `dispatch_result` contains an `sdlc_result` dict (meaning SDLC ran), the
outer `run_pipeline()` was overwriting the SDLC's `pipeline_status` with
`dispatch_failed` whenever `pipeline_success` was `False`. This lost the actual
SDLC status — `tests_failed` (code produced but tests couldn't run in the env)
was indistinguishable from a real dispatch failure. The pipeline reported
failure even though working code was generated.
**Fix (P16, Jun 2026):** Check for `sdlc_result` in `dispatch_result`. If
present, propagate `sdlc_result['pipeline_status']` and
`sdlc_result['pipeline_success']` to the outer result. Special-case
`tests_failed`: set `pipeline_success=True` and `error=None` because code WAS
produced — the test environment just wasn't available. Other SDLC statuses
(`success`, `council_reviewed`, `debug_failed`) propagate as-is.
**Verification:** 5 ad-hoc monkeypatched scenarios (tests_failed, success,
council_reviewed, debug_failed, non-SDLC fallback) + 92 unit tests in
`test_pipeline.py`. Live E2E fibonacci test confirmed the fix: code produced
and executed correctly even when pytest wasn't available in the environment.
GLM-5.2 defaults to Chinese. The script auto-appends "respond in English only"
for known non-English models.
The parser stops collecting aliases at the first unrecognized word. If the
user says "ask deepseek what is...)", "what" is not an alias, so it becomes
part of the prompt. This is correct behavior.
Sessions are saved to `~/.hermes/ask-sessions.json` on the host running the
script. They persist across restarts but are not shared across machines.
All writes to the session registry use atomic `tempfile + os.replace()` to
prevent JSON corruption from parallel `ask` calls (save_session, _remove_session,
and clean_expired_sessions all use this pattern).
If a saved session ID no longer exists in Hermes' session store (e.g., the
session was pruned, the ID was test garbage, or the database was reset),
`dispatch_single()` detects "Session not found" in stderr and automatically:
1. Strips `--resume <id>` from the command
2. Removes the stale alias from `ask-sessions.json` via `_remove_session()`
3. Retries the call fresh (no resume)
4. Sets `resume_session = None` so the new session saves correctly
This means `ask kimi "prompt"` will never fail with "Session not found" —
it falls back to a fresh session and cleans up the stale entry. The user
sees a normal response; the cleanup is silent.
### Pipeline alias resolution (pipeline.py)
The SDLC pipeline (`pipeline.py`) calls `routing.route()` which returns
**aliases** (e.g., `"deepseek"`, `"fast"`, `"glm"`), not full model names.
But `hermes chat -m` requires full model names (e.g., `"deepseek-v4-pro:cloud"`).
The pipeline MUST call `resolve_alias()` on the routed model before passing
it to `dispatch_single()`. Without this, live pipeline calls fail because
Hermes doesn't resolve the alias. This was discovered and fixed Jun 2026
(commit 2bd9835). Tests in `TestPipelineAliasResolution` verify all 3
cost-tier aliases resolve correctly.
In comparison mode (multiple models), each call is independent — no session
is saved. Only single-model calls save sessions for follow-up.
When `dispatch_comparison` calls `dispatch_single`, all optional parameters
must be passed as **keyword arguments** (e.g., `thinking=thinking`), not as
positional arguments. Tests mock `dispatch_single` and assert on
`call_args.kwargs`, so positional args cause test failures even when runtime
behavior is identical. Discovered Jun 2026 when consolidating
`dispatch_comparison` from ask.py into model_utils.py — the model_utils
version passed `thinking` positionally, causing 2 test failures in
`TestDispatchComparisonThinkingKwarg`.
When a function is moved from one module to another (e.g., `dispatch_comparison`
from ask.py → model_utils.py), audit the source module for imports that were
only used by the moved function. `concurrent.futures` was imported in ask.py
solely for `dispatch_comparison`'s `ThreadPoolExecutor` — after the move, it
was dead code. Run `grep` for each import to confirm it still has callers
before leaving it in place.
### Session ID is on stderr, not stdout
When `hermes chat -q` runs in quiet mode, the session ID is printed to
**stderr**, not stdout. The `ask.py` script captures both streams and extracts
the session ID from stderr. If you're calling `hermes chat -q` directly (not
through `ask.py`), remember to capture stderr:
```python
r = subprocess.run(cmd, capture_output=True, text=True)
Batch model assignment needs DeepSeek review
When assigning models to multiple cron jobs (batch reassignment), the fast
model (qwen3.6) makes reasonable-but-wrong assignments. It correctly identifies
simple vs complex tasks but misses domain-specific nuances: GLM defaults to
Chinese, user-facing briefings need quality, watchdogs that run every 30m
should use the cheapest model that can do the job. Always send the full job
context to DeepSeek for review before committing batch model changes. See
references/cron-model-tiers.md for the tier system and decision framework.
Prompt with -- or special characters
If the user's prompt contains -- or flag-like text (e.g., "What does
--max-turns mean?"), use the --prompt flag instead of positional args.
The script uses parse_known_args() to handle this, but --prompt is
always safer for prompts with special characters.
QA Code Review
Use ask qa to run a structured code review via the local Qwen coder model:
python3 ask.py qa -p "Review routing.py, model_utils.py, and triage.py for bugs" \
--mode agent --thinking low --max-turns 15 --timeout 180 \
--toolsets file,terminal
Prompt template for structured reviews:
Review the following Python files for bugs, race conditions, inconsistencies,
security issues, and edge cases:
1. /path/to/file1.py
2. /path/to/file2.py
Output a structured report with sections: BUGS, RACE CONDITIONS, INCONSISTENCIES,
SECURITY, PERFORMANCE, EDGE CASES. For each finding note severity (HIGH/MEDIUM/LOW),
file, line reference, and brief fix. If no issues in a section, write "No issues found."
Be concise but thorough. Only report real issues — no style nitpicks.
Why agent mode, not raw: QA review needs the model to read files, search for
patterns, and cross-reference between files. Raw mode truncates at ~2K chars and
can't access the codebase. Agent mode gives the model file and terminal
toolsets so it can read the actual code and verify its findings.
Performance: ~80s for 3 files (~10K lines total) on qwen3-coder-next:q4_K_M.
The model found 9 HIGH, 15 MEDIUM, 13 LOW issues across routing.py, model_utils.py,
and triage.py — including a real timeout default mismatch and missing input
validation that would have caused production failures.
Pitfall: Raw mode truncates output at ~2K chars — not enough for a full review.
Always use --mode agent for QA reviews. Raw mode is for classification and
simple Q&A only.
DeepSeek Full-Codebase Audit
For a comprehensive audit of an entire skill or codebase (not just specific
files), dispatch DeepSeek at --thinking high with a structured audit prompt
and file+terminal toolsets. The model reads every file in the directory and
produces a structured report covering features, bugs, and documentation gaps.
When to use: After building a new feature or making significant changes to a
skill/codebase, a full audit catches bugs, missing tests, undocumented features,
and documentation drift that file-by-file QA reviews miss.
Prompt template for full-codebase audits:
python3 /opt/data/skills/productivity/ask/scripts/ask.py deepseek \
--prompt "You are auditing the <skill-name> skill at /opt/data/skills/<category>/<skill-name>/.
Read ALL files in the directory — scripts/, tests/, SKILL.md, pyproject.toml, and any
reference files. Produce a structured audit report with these sections:
A. FEATURE INVENTORY — every function across all scripts, with file and line
B. BUGS — any correctness issues, grouped by severity (HIGH/MEDIUM/LOW)
C. DOCUMENTATION GAPS — features in code but missing from SKILL.md
D. TEST COVERAGE GAPS — functions with no test coverage
E. CONFIGURATION ISSUES — missing dependencies, wrong paths, stale references
For each finding, note the file, line reference, severity, and a one-sentence fix.
If a section has no issues, write 'No issues found.' Be concise but thorough.
Only report real issues — no style nitpicks." \
--thinking high --timeout 300 --toolsets file,terminal \
-o /tmp/<skill-name>-audit.md
Performance: ~84s for a 10-file, 200-test skill (USAW event info, 2026-07-11).
DeepSeek at --thinking high produced a thorough 37-function inventory with only
1 MEDIUM bug and 13 LOW observations — the codebase was clean.
After the audit:
- Read the report (
read_file /tmp/<skill-name>-audit.md)
- Fix any HIGH/MEDIUM bugs immediately
- Add missing tests for uncovered functions
- Update SKILL.md with undocumented features and test counts
- Commit everything with THESIS/LEARNINGS/REFERENCES citing the audit
Why DeepSeek, not Kimi or Qwen: DeepSeek V4 Pro at --thinking high produces
the most thorough codebase-level analysis — it reads every file, cross-references
functions across modules, and catches documentation drift. Kimi is better for
targeted code review of specific files; DeepSeek is better for holistic audits
that require understanding the full codebase architecture.
QA Fix Workflow
After a QA review surfaces issues, follow this pattern (demonstrated Jun 2026
on triage/routing/model_utils/ask — 11 fixes across 4 files, 115/115 tests pass):
- Verify findings — read the actual lines the QA flagged; some may be
false positives or already correct.
- Write a fix plan — table with file, issue, fix, and line references.
Present to user before editing.
- Batch fixes — apply all fixes in one pass, grouped by file.
- Verify — compile all changed files, run ad-hoc checks on each fix
(input validation, cache behavior, error handling), run full test suite.
- Commit — one commit with all fixes, descriptive message listing each
fix by severity and file.
Verification discipline: The system tracks changed paths and flags
"unverified" after a commit even when verification was already done
pre-commit. After committing, re-run a quick ad-hoc verification (compile
all changed files + run test suite) to satisfy the flag. This is a
mechanical re-check, not a new verification — the actual verification
happened in step 4. Pattern: verify → commit → system flags unverified →
re-verify (quick compile + test suite).
Key insight: QA reviews find real bugs but also false positives. Always
verify line references before editing. The QA model (qwen3-coder-next) found
10 HIGH issues — all 10 were real and fixable. But it also flagged some MEDIUM
issues that were already correct (e.g., Optional import was present, just
not on the line the QA expected).
Config Deference: Don't Override Hermes Config — ANY Parameter
Skills should NOT impose their own limits for ANY parameter that Hermes already
has a config key for. Default to None/omit and let Hermes config be the source
of truth. Only hardcode when the skill has a domain-specific reason (e.g., SDLC
text-output phases need max_turns=1 to prevent tool calls).
This applies to ALL parameters, not just max_turns:
| Parameter | Hermes Config Key | Hardcoded Example | Fix |
|---|
max_turns | agent.max_turns (default: 120) | max_turns=5 | max_turns=None |
thinking | agent.reasoning_effort (default: medium) | thinking="low" | thinking=None |
model | agent.model | hardcoded model name | use alias resolution |
timeout | (no config key — safety net, OK to hardcode) | timeout=3600 | keep as safety net |
Real example 1 — max_turns (Jun 2026): The ask skill hardcoded
DEFAULT_MAX_TURNS = 5, always passing --max-turns 5 to hermes chat. The
user's Hermes config had agent.max_turns: 120. The skill was silently capping
every ask call at 5 turns. Fix: Optional[int] = None in all signatures, only
pass --max-turns when explicitly set. User's correction: "Hermes already has
max_turns: 120 in config — ask.py shouldn't override that with its own lower
default."
Real example 2 — thinking (Jun 2026): The SDLC orchestrator hardcoded
thinking="medium" for planner, thinking="low" for coder, thinking="high"
for verifier — 6 dispatch sites + 2 cascade entries. The user's correction:
"No, don't hardcode thinking levels either. Pass None and let Hermes config be
the source of truth." Fix: all thinking= values → None. The dispatch_single
function already skips config mutation when thinking is None, so the model
inherits whatever agent.reasoning_effort is set to (or the model's default).
Pattern: Optional[X] = None → omit flag/config mutation when None →
Hermes config wins. Only hardcode parameters that have NO Hermes config key
(e.g., timeout as a safety net, toolsets as a per-role access control).
timeout applies to each dispatch attempt. With retries, the pipeline dispatch stage
can take up to (max_retries+1) × timeout wall-clock time.
Audit technique: When the user flags one hardcoded parameter, audit ALL
parameters across ALL dispatch sites in ALL files — not just the one they
flagged or the file you were working in. The user flagged max_turns in
sdlc_state.py; the follow-up audit found thinking hardcoded at 6 more
sites in the same file, then a second sweep found 6 more sites in sdlc.py
that the first audit missed. A single grep for dispatch_single( across
the entire codebase, then tracing each call's kwargs, catches everything
in one pass. Don't stop at one file.
Don't Override --timeout for Complex Editing Tasks
The --timeout flag on ask.py caps the subprocess wall-clock time. For complex
multi-file editing tasks (3+ files, 10+ distinct fixes), the default 3600s is
appropriate — do NOT pass a lower --timeout. A 300s timeout will kill the agent
mid-edit, losing the response summary even though file edits were already saved.
Real example (2026-06-29): Kimi was dispatched to apply 11 fixes across 3
SDLC files (~5,400 lines). --timeout 300 was passed explicitly, overriding the
60-minute Hermes config default. Kimi completed all file edits but hit the 300s
limit while writing its response summary. The edits were saved (verified by
git diff), but the summary was lost. Lesson: for complex editing tasks, omit
--timeout entirely and let Hermes config (3600s) be the limit. Only set
--timeout for quick lookups or when you have a specific reason to cap runtime.
Default Changes Must Audit All Entry Points
When changing a default value that appears in multiple files (e.g., max_turns
default in ask.py, model_utils.py, AND pipeline.py), audit every CLI entry
point and function signature in the codebase. The ask skill has three independent
entry points that each carry their own copy of the default:
| File | Entry point | Has own default? |
|---|
ask.py | argparse --max-turns | Yes |
model_utils.py | dispatch_single(max_turns=...) | Yes (signature default) |
pipeline.py | run_pipeline(max_turns=...) + argparse --max-turns | Yes (TWO copies) |
prompt_model.py (advisors skill) | argparse --max-turns | Yes |
Real example (Jun 2026): DEFAULT_MAX_TURNS was removed from model_utils.py
and ask.py was updated to default=None, but pipeline.py still had
max_turns: int = 5 in run_pipeline() and default=5 in its argparse. The
pipeline would silently override the user's Hermes config with 5 turns.
Fix: After any default change, grep for the old value across ALL scripts:
grep -n "max_turns.*5\|DEFAULT_MAX_TURNS" scripts/*.py
Stale test assertions after alias/model changes
When an alias in model_utils.py changes (e.g., "fast" from gemma4:12b-mlx-bf16
to qwen3.6:35b-a3b), any test that asserts the old model name in output will
fail. The test is stale — it was correct when written but the alias mapping
changed underneath it.
Real example (Jun 2026): test_cli_file_output asserted gemma4 in the
output when using the "fast" alias. The alias was changed to qwen3.6:35b-a3b
in a prior commit. The test failed with AssertionError: 'gemma4' not found in output — the output now contained qwen3.6:35b-a3b.
Detection: After any alias change in model_utils.py, grep for the old
model name in the test files:
grep -rn "gemma4" tests/
Fix: Update the test assertion to expect the new model name. This is not a
bug — the test was correct for the old alias and needs updating for the new one.
Mocking functions with internal try/except: use return_value, not side_effect
When mocking a function that has its own try/except block (like
_fuzzy_resolve_raw which catches exceptions and returns None), do NOT use
side_effect=Exception(...) — the exception raises at the mock call site
before the real function's try/except can catch it. Use return_value=None
to simulate the function returning None after catching the exception internally.
@patch("model_utils._fuzzy_resolve_raw", side_effect=Exception("network error"))
def test_network_error(self, mock_raw):
...
@patch("model_utils._fuzzy_resolve_raw", return_value=None)
def test_network_error(self, mock_raw):
...
Real example (Jun 2026): test_fuzzy_llm_exception_returns_original used
side_effect=Exception("network error") to test graceful failure. The test
failed because the exception raised at the mock boundary, never reaching
_fuzzy_resolve_raw's internal try/except. Fixed by changing to
return_value=None — the function's real implementation catches exceptions
and returns None, so the mock should return None directly.
assert_called_with vs call_args_list for multi-call mocks
When a function calls a mock multiple times (e.g., set_reasoning_effort('high')
then set_reasoning_effort('medium') to restore), assert_called_with only
checks the last call. Use call_args_list to check all calls:
mock_set.assert_called_with("high")
call_args = [c.args[0] for c in mock_set.call_args_list]
self.assertIn("high", call_args)
Real example (Jun 2026): council_review() calls set_reasoning_effort('high')
before dispatch and set_reasoning_effort(original) after. The test used
assert_called_with('high') which failed because the last call was the restore.
Fixed by checking call_args_list for "high" at any position.
Ad-Hoc Verification Script Gotchas
When writing ad-hoc verification scripts under /tmp/hermes-verify-*.py, these
patterns cause false failures and wasted iterations:
1. grep -n line-number prefix breaks comment detection
grep -n output includes the line number: 1255: # comment. Calling
.strip() on this string does NOT produce # comment — it produces
1255: # comment. So stripped.startswith("#") is always False.
Fix: strip the NNNN: prefix with re.sub(r'^\d+:\s*', '', line) before
checking if the remaining text is a comment.
2. grep -c with multiple files returns per-file counts
grep -c PATTERN file1.py file2.py returns file1.py:3\nfile2.py:0, not 3.
int(r.stdout.strip()) raises ValueError. Fix: grep each file separately,
or use grep -c PATTERN file1.py individually.
3. Indented comments are not detected by line.strip().startswith("#")
A line like # outputs text, not tool calls has leading whitespace
before the #. line.strip() on the grep output (which includes the NNNN:
prefix) doesn't help. After stripping the line-number prefix, code_part.strip()
correctly identifies it as a comment.
4. Passthrough kwargs look like hardcoded values
thinking=thinking and max_turns=max_turns are passthrough variables, not
hardcoded levels. Regex checks for thinking='medium'/'low'/'high' must
exclude these. Also exclude log lines like f"thinking={event.get('thinking')}".
5. Verification scripts often need 3-5 iterations
The first pass almost always has a regex bug. Expect to iterate. Clean up
/tmp/hermes-verify-*.py files after the final pass.
Discovered Jun 2026 during the hardcoded-values audit verification (5 iterations:
v1 had grep -c multi-file bug, v2 had passthrough false positive, v3 had
indented-comment false positive, v4 had line-number prefix bug, v5 passed 16/16).
--cwd and --emit-events: New CLI Flags (Jun 2026)
Two new CLI flags added as part of the "dumb pipe" improvement plan:
--cwd — Sets the working directory for the dispatched agent. Forwarded to
dispatch_single(cwd=...). Only works in single-model mode; prints a warning
and is ignored in comparison mode (no effect on multiple models). Use for
worktree-isolated dispatch:
ask deepseek "Review this code" --cwd /opt/data/projects/my-feature
--emit-events — Prints JSON dispatch events to stderr for programmatic
callers. Uses _make_stderr_event_callback() factory. stdout output is
unchanged — events are sideband on stderr:
ask deepseek "Design API" --emit-events 2>events.jsonl
Verification Re-Run After Commit (System Flag)
The system tracks changed paths and flags "unverified" after a commit even when
verification was already done pre-commit. After committing, re-run a quick
ad-hoc verification (compile all changed files + run test suite) to satisfy the
flag. This is a mechanical re-check, not a new verification — the actual
verification happened pre-commit. Pattern: verify → commit → system flags
unverified → re-verify (quick compile + test suite). Using python3 -c inline
avoids temp file cleanup and is faster than writing a /tmp/hermes-verify-*.py
script for the re-run.
Sibling Subagents Can Revert File Changes
When multiple subagents are working in the same session (e.g., dispatched via
delegate_task), a sibling subagent can overwrite or revert file changes made
by the parent session. This happened Jun 2026: the DEFAULT_MAX_TURNS constant
was removed from model_utils.py, but a sibling subagent (20260628_021420_e7fe93)
reverted the file, restoring the old constant. The parent session's subsequent
verification caught it (grep found the constant still present).
Mitigation (detect): After any subagent dispatch that touches files you've
edited, re-verify your changes are still in place before declaring the task
done. A quick grep for the key change is sufficient.
Mitigation (lock in): If a sibling reverts your changes more than once,
commit and push immediately to lock them in git. Sibling subagents can't
revert committed changes without a conflicting commit — they can only
overwrite uncommitted working-tree files. The user's directive "Apply and
commit now" (Jun 2026) was the correct recovery: git add <files> && git commit && git push. After the commit, the sibling stopped reverting.
This is distinct from the "verify commits" pitfall below (which covers
subagents making unsanctioned changes) — this covers subagents reverting
changes the parent already made.
Subagent Code Reviewers: Verify Commits Before Accepting
When dispatching a code reviewer via delegate_task (especially qwen-coder),
the subagent may make unsanctioned changes beyond the review scope. In
Jun 2026, a qwen-coder subagent dispatched to review alias changes made three
unauthorized modifications:
- Changed
debugger alias from kimi-k2.7-code:cloud to qwen3-coder-next:q4_K_M
- Added
debugger-fallback and test-planner aliases not in the codebase
- Gutted
sdlc-plan-2026-06-27.md from 156 lines to 46 lines
Mitigation: After a subagent code review completes, always:
git diff HEAD~1 to see what the subagent actually committed- Revert any changes outside the review scope
- Verify alias tables in SKILL.md match
model_utils.py ALIASES dict
- Check reference files weren't truncated or rewritten
Subagent Escaping Artifacts: Double-Escaped Strings in Code
Subagents (especially qwen-coder) can produce double-escaped strings when
writing code — literal \\\\n instead of \n, and f\\" instead of f".
These are NOT valid escape sequences; they're literal backslashes in the source
that produce broken strings at runtime.
Real example (Jun 2026): A qwen-coder subagent fixing sdlc.py bugs wrote:
prompt = f"Fix the code:\\\\n\\\\n{code}\\\\n\\\\nError: {error}"
msg = f\\"Timeout after {elapsed}s\\"
The patch tool applied these as-is because they're valid Python syntax
(strings containing literal backslash characters). The code compiled but
produced garbled output at runtime.
Detection:
grep -n '\\\\\\\\n' skills/productivity/ask/scripts/sdlc.py
grep -n 'f\\\\"' skills/productivity/ask/scripts/sdlc.py
Fix: Replace \\\\n → \n and f\\" → f" in the affected lines.
These are always artifacts — there is no legitimate reason for \\\\n
inside a Python string literal in this codebase.
Prevention: Include in subagent context: "Do not double-escape backslashes
in string literals. Write \n not \\n, and f\" not f\\\"."
set_reasoning_effort Race in Parallel Dispatch
When multiple threads call hermes config set agent.reasoning_effort <level>
in parallel (e.g., council_review() dispatching 3 models simultaneously),
they race on the global config file. Thread A sets high, Thread B sets high
(overwriting A's save of the original), Thread A restores "original" (which is
now B's high), Thread C sets high (overwriting A's restore), etc. The
result is unpredictable — some models get wrong thinking levels, and the final
restored value may be wrong.
Fix: Set reasoning_effort ONCE before the parallel block, restore ONCE
after. Remove per-thread thinking='high' kwargs. The global config is the
single source of truth for the duration of the parallel block.
Real example (Jun 2026): council_review() in sdlc.py had each of 3
_dispatch_seat() calls doing its own set_reasoning_effort('high') +
finally: restore. DeepSeek architectural review caught this — the 3 threads
stomped on each other's config saves. Fixed by setting once before
ThreadPoolExecutor and restoring once after.
Dead Code: Verify New Functions Have Callers
When adding a new function, verify it's actually called somewhere. It's easy
to define a cleanup/helper function and forget to wire it into the call path.
Real example (Jun 2026): _cleanup_sdlc_sessions() was defined in sdlc.py
to remove stale __sdlc_* session entries, but was never called. Stale sessions
from crashed/aborted pipelines persisted for up to 1 hour. DeepSeek architectural
review caught this — the function existed but had zero callers. Fixed by adding
a call at the start of run_test_first_pipeline().
Mitigation: After adding a new function, grep for its name across the
codebase to confirm it has at least one caller outside its own definition.
# Single
ask <alias> <question>
ask deepseek What is ACID compliance?
# With thinking level
ask deepseek "Prove this is O(n log n)" --thinking xhigh
ask fast "What is 2+2?" --thinking none
# Comparison
ask <alias1> <alias2> <question>
ask deepseek kimi Should we use PostgreSQL or MongoDB?
# Follow-up (auto-resumes)
ask deepseek Now add WebSocket support
# File output
ask deepseek Plan the architecture -o /tmp/plan.md
# Sessions
ask --sessions
ask --session deepseek
References
Running Tests
Tests use pytest (not installed by default — use uv run):
cd /opt/data/skills/productivity/ask && uv run --with pytest python3 -m pytest tests/ -v
cd /opt/data/skills/productivity/ask && uv run --with pytest python3 -m pytest tests/ -v -k "not live"
cd /opt/data/skills/productivity/ask && uv run --with pytest python3 -m pytest tests/test_ask.py::TestNeedsNoThink -v
Tests (340 pytest-collected): 166 in test_ask.py, 17 in
test_contract.py, 6 in test_gate_driver.py, 116 in test_pipeline.py, 11
in test_pipeline_e2e.py, and 24 in test_routing.py. The opt-in,
shell-driven live corner-case suite is separate: tests/live/TEST_PLAN.md
defines its eleven LC cases and tests/live/run_live_suite.sh runs them and
appends dated results to that plan. It exists because it caught a real
0700-working-directory PermissionError platform bug and an enum-normalization
gap.
Pipeline Architecture
The retired sdlc.py 11-phase engine was removed on 2026-07-01. The live
pipeline is deliberately smaller:
User message
│
▼
triage.py → routing.py → single dispatch
└→ devloop via devloop_bridge (test_first / debug_cascade)
triage.py classifies the request and routing.py selects the skill, model,
thinking level, toolsets, and (where applicable) pipeline mode. Ordinary
routable work reaches dispatch_single(); build and debug modes use a live,
enabled devloop_bridge route instead.
That devloop handoff is an intentional fail-closed three-way split:
- If importing
devloop_bridge failed, a test_first or debug_cascade
request receives a failed dispatch_result and exit 1; it must not
silently degrade to an unverified single-shot answer.
- If
DEVLOOP_ENABLED=0, the operator-selected kill-switch intentionally
clears the pipeline mode and uses ordinary single-shot dispatch.
- If devloop is live and enabled,
test_first calls
devloop_bridge.call_guarded(devloop_bridge.run_build, ...) and
debug_cascade calls
devloop_bridge.call_guarded(devloop_bridge.run_debug, ...), both with the
scratch workspace. Their outcome is classified by devloop's shared classifier.
11 categories: query_model, build_code, debug_code, research_info, urgent_action, general_chat, deploy_code, write_docs, config_change, status_check, explain_concept
Cost tiers:
| Budget | Models | Use case |
|---|
free | fast, qwen, gemma | Local models (no API cost) |
low | glm, kimi | Cheap cloud (per-token pricing) |
medium | deepseek, minimax | Mid-tier cloud |
high | deepseek, kimi | Multi-model consensus |
references/lean-dispatch.md — Slim Hermes agent: --ignore-rules + limited toolsets for ~40% speedup (Jul 2026)references/local-model-turn-benchmarks.md — Per-model turn times and practical max-turn recommendations (Jun 2026)references/fuzzy-alias-resolution.md — Two-tier fuzzy alias resolution: architecture, prompt design, API, test coverage (Jun 2026)scripts/ask.py — The improved prompt script with aliases, sessions, comparison mode, --mode raw, and --clean-sessionsscripts/pipeline.py — triage → routing → {single dispatch | devloop} pipeline. CLI entry point with --dry-run, --json, --cost-budget modesscripts/model_utils.py — Shared utilities: build_prompt(), dispatch_single(), clean_expired_sessions(), SESSION_TTLscripts/routing.py — Triage-to-dispatch routing layer with cost tiers, LRU caching, and pipeline event logging
