菜单
Research and compare AI agent harnesses (coding/automation agents like Claude Code, Pi, OpenCode, OpenClaw, Hermes Agent) with model compatibility analysis. Covers the "Harness Effect" — how the same model performs 5-40 percentage points differently depending on the harness.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
Comprehensive wiki health checking, maintenance, and remediation — duplicate detection, entity dedup/disambiguation, link auditing/fixing, page splitting, source-linking lint, wikilink remediation, tag taxonomy audit and normalization, pre-commit enforcement, language enforcement (JP→EN bulk translation, detection regex, cron-assisted migration), and decision-matrix-driven cleanup.
Comprehensive wiki maintenance: daily structural health checks (index reconciliation, log separator fixes, pipeline watchdog alerts), comparison page updates (adding items to multi-section comparison tables), and page relocation (moving/renaming pages while maintaining link integrity).
Pre-flight checklist and procedures for archiving, deleting, or migrating Hermes skills. Prevents accidental removal of cron-referenced skills. Covers the 3-layer skill structure and config.yaml management. Includes skill inventory management, promotion workflows, and archival conventions.
Karpathy's LLM Wiki: build/query interlinked markdown KB.
Deep analysis of blog authors' recent thoughts, philosophy, and positions. Goes beyond entity page creation to extract cited ideological positions, track thought evolution, and enable ongoing RSS monitoring for thought updates.
Query the blogwatcher-cli SQLite database for RSS scan results. Use pre-verified column names and query templates to avoid errors.
| name | agent-harness-research |
| description | Research and compare AI agent harnesses (coding/automation agents like Claude Code, Pi, OpenCode, OpenClaw, Hermes Agent) with model compatibility analysis. Covers the "Harness Effect" — how the same model performs 5-40 percentage points differently depending on the harness. |
| trigger | When the user asks about: - "ハーネス" (harness) in context of AI agents/coding tools - Which agent harness works best with which model - Harness vs harness comparisons (e.g., "Pi vs OpenCode with Qwen") - "Harness Effect" or model-harness compatibility - Benchmarks across coding agents - agent CLI tools (Claude Code, Codex CLI, OpenCode, Pi, Aider, Cursor) - "HarnessとFramework/SDKの違い" (difference between harness and framework/SDK) — see §3b - "どのagent frameworkを選ぶべきか" (which agent framework to choose) - Agent infrastructure investment decisions (operator workbench vs product runtime) Critical trigger: User says "ハーネス" — ALWAYS interpret as AGENT HARNESS (Claude Code, Pi, OpenCode, etc.), NOT inference server (vLLM, llama.cpp, Ollama). The user's domain is coding/automation agent harnesses. Secondary trigger: User asks about harness engineering in domain-specific contexts (AEC, engineering report generation, non-coding agent systems). Theodoros Galanos' work on The Harness Blog demonstrates harness engineering extends beyond coding agents into knowledge-work domains. |
In this user's context, "harness" = AI Agent Harness (coding agent / automation agent framework), NOT inference server.
| ✅ Agent Harnesses | ❌ Not Agent Harnesses |
|---|---|
| Claude Code, Codex CLI, OpenCode, Pi | vLLM, llama.cpp, Ollama (these are inference engines) |
| OpenClaw, Hermes Agent, Aider, Cursor | SGLang, TensorRT-LLM (these are model servers) |
| Cline, Windsurf, Continue | API providers (OpenAI, Anthropic, Google) |
Why this matters: If the user asks about "ハーネス" and you start talking about vLLM vs llama.cpp, you will get a correction. The user's interest is in coding agent frameworks that harness LLMs to write/edit/run code autonomously.
Harness vs Runtime distinction (dual-perspective):
Within the agent stack, the harness and the runtime are distinct layers. The wiki now recognizes two complementary views of the runtime:
| View | Runtime as... | Key concern | Source |
|---|---|---|---|
| Infrastructure substrate | Compute, filesystem, network boundary, isolation primitives | Where and on what the agent runs | Han Lee (Apr 2026) |
| Execution semantics | Lifecycle controller, tool mediator, state keeper, scheduler, event emitter, safety enforcer | How execution proceeds safely and continuously | kzinmr (May 2026) |
The harness decides what the agent attempts (orchestration: tools, prompts, loops, context, memory, verification). The runtime manages how execution proceeds (lifecycle, tool mediation, state continuity, scheduling, events, safety, observability). This makes the runtime analogous to an OS kernel — it manages process lifecycle, scheduling, I/O mediation, permissions, eventing, and state, but at the agent execution layer rather than the hardware layer.
Han Lee's formulation: "The agent is the harness plus the model, running inside the runtime."
Distinction from workflow frameworks: LangGraph is closer to a workflow framework — it describes execution topology (what should happen). Claude Agent SDK and PI are closer to runtimes — they maintain execution continuity (how execution proceeds).
When the user asks about agent infrastructure, sandboxing, runtime architecture, or the harness/runtime boundary, also consult [[concepts/agent-runtime]] (especially §"Execution Semantics: The Control System Layer") and [[concepts/agent-harness]] (§"Harness vs Runtime: The Critical Distinction").
The single most important finding in this domain: the harness is as important as the model.
Same model, different harness:
General finding: Multiple independent studies show the harness effect ranges from 5 to 40 percentage points depending on model and task type.
The runtime-centric family (kzinmr, 2026-05-15): ClaudeCode, Codex CLI, PI, OpenClaw, and Hermes Agent are all runtime-centric systems in the same architectural family — they manage how execution proceeds rather than describing what execution topology should be (workflow-centric). LangGraph and PydanticAI are workflow-centric. The key distinction is not "closed vs open" but "runtime-centric vs workflow-centric." See [[concepts/agent-runtime]] §"Execution Semantics: The Control System Layer" and [[comparisons/open-harness-vs-agent-framework]] §9.
Root causes of harness effect:
The Harness Effect is not limited to coding agents. Theodoros Galanos demonstrated it in the AEC domain with identical models:
| Metric | Open REPL Harness | Lambda-RLM Harness | Improvement |
|---|---|---|---|
| Total Tokens | 740K | 53K | 14x less |
| Quality (Reward) | 0.67 | 0.73 | +8.4% |
Core thesis: "The agent's reasoning architecture is itself a harness design choice. Agent design and harness design are the same problem." — Galanos argues that the line between model-level reasoning architecture (open REPL vs deterministic pipeline) and harness design dissolves completely.
Implication: Harness engineering applies broadly beyond coding agents to any domain-specific agent system (AEC reports, HVAC analysis, document generation). The harness effect magnitude (5-40pp for coding agents, 14x token/8.4% quality for AEC) is comparable or larger in domain-specific work.
Major agent harnesses to track as of May 2026. Full wiki entity pages exist for most — see [[concepts/agent-harness-comparison]] for the 9-harness comparison portal.
| Harness | Type | Creator/Org | Model Policy | Wiki Entity |
|---|---|---|---|---|
| Claude Code | Agent orchestrator | Anthropic→OpenAI | Anthropic models ONLY (Opus 4.7, Sonnet 4.6). Subscription $20/mo | [[entities/claude-code]] |
| OpenCode | Model-agnostic harness | AnomalyCo/SST | 155K GitHub stars. 75+ providers; Claude/GPT/Gemini/Grok/local. BYOK or $10/mo OpenCode Go | [[entities/opencode]] |
| Pi | Minimalist harness | Mario Zechner | 45.5K GitHub stars. 20+ providers; optimized for local (MLX/GGUF). BYOK. Anthropic wall: Claude Max subscription NOT usable in Pi — double-billing | [[entities/pi]] |
| Codex CLI | Universal coding agent | OpenAI (Apache-2.0 OSS, 79.3K GitHub stars, Rust 96.2%) | GPT-5.5 (recommended), GPT-5.4, GPT-5.3-Codex-Spark, GPT-OSS (local), custom providers (DeepSeek, Qwen, etc. via config.toml), local (Ollama/LM Studio/MLX). Included in ChatGPT Plus/Pro/Team/Enterprise ($20-200/mo). MCP dual, sub-agents, image gen, remote TUI, code review. Most underrated harness — commonly misreported as single-model/closed-source. | [[entities/codex]] |
| Copilot CLI | GitHub-native agent | GitHub/Microsoft | 6 built-in sub-agents (explore/code-review/research/general/rubber-duck/configure). /fleet parallel. MCP-powered. BYOK + local models (since Apr 2026). Included in Copilot Free/Pro/Pro+/Business/Enterprise. | [[entities/copilot-cli]] |
| Droid | Enterprise multi-platform | Factory AI | CLI/IDE/Slack/Linear/CI/CD. 3-level Auto-Run. Specialized sub-agents (CodeDroid/Review Droid/QA Droid). SOC-2, SSO. $20-50+/mo. | [[entities/droid]] |
| Cursor | IDE-integrated | Cursor | Claude Opus, GPT-5, custom. Subscription $20-$200/mo | — |
| Kilo | All-in-one platform | Kilo Org (Apache-2.0) | OpenCode fork with VS Code + JetBrains + CLI. 500+ models via Kilo Gateway (zero markup). Hosted OpenClaw (KiloClaw). Teams/SSO/Analytics. Inline autocomplete, Cloud Agents, code review. Free tier + pay-as-you-go. | [[entities/kilo]] |
| OpenClaw | Always-on Telegram agent | Peter Steinberger / NVIDIA | Built on Pi SDK. Local via Ollama/LM Studio, or any API. MIT license. 145K+ GitHub stars. | [[entities/openclaw]] |
| Hermes Agent | Self-improving persistent agent | Nous Research | Multi-model via config; DeepSeek V4, Gemini, Claude, Ollama. Open-source | [[entities/hermes-agent]] |
| Aider | Git-first harness | Independent | BYOM (any model). Token-efficient (1/4.2 of Claude Code) | — |
When comparing harnesses, consider these dimensions:
references/openclaw-orchestrator-evidence-deck.md — condensed 8-source evidence deck supporting OpenClaw as orchestrator.\n- popularaitools.ai — Hermes Agent vs OpenClaw — Confirms speed advantage ("significantly faster than OpenClaw on the same model"), dual-agent recommendation (ACP bridge), and practical multi-agent workflows. Source: https://popularaitools.ai/blog/hermes-agent-vs-openclaw\n- John Berryman (Arcturus Labs) — Unharnessed Agents: The anti-harness thesis. Berryman argues "agent harness" is the wrong frame entirely: the agent IS the whole composition (while loop, LLM calls, tool calls, context management, skills, MCP). Agents should leave the IDE and become personal assistants, interface builders, workflow operators. Skills are the new programs; English is the new programming language; agents are the new runtime. We need standardized, pluggable agent primitives (Lego blocks). Source: Unharnessed Agents Power the Future of AI Products (April 2026). Wiki entities: [[entities/john-berryman]], [[concepts/unharnessed-agents]]Two complementary theses define the poles of the harness debate. When researching harness topics, always consider both:
| | Amp / [[entities/thorsten-ball|Thorsten Ball]] | Berryman / [[entities/john-berryman|Arcturus Labs]] | |---|---|---| | Core claim | Harness differentiation is dead — models absorb harness features | "Harness" is the wrong frame — call them agents | | Direction | Harness → Model (vertical: models eat harnesses from above) | Harness → Agent (horizontal: rename and expand scope) | | What dies | The harness as competitive moat | The term "harness" and IDE-centric mindset | | What comes next | Codebase organization, model-native features | Standardized agent primitives, agents as universal runtime | | Convergence | ✓ Current harness framing is limiting | ✓ Harnesses are too opinionated/opaque | | Wiki page | [[concepts/harness-commoditization]] | [[concepts/unharnessed-agents]] |
Both imply that today's harness products (Claude Code, Cursor, Codex, OpenCode) are transitional forms — not the end state.
When the user asks about "HarnessとFrameworkの違い" (difference between Harness and Framework/SDK), consult this section AND the wiki's comprehensive comparison at [[comparisons/open-harness-vs-agent-framework]] (created 2026-05-14 from kzinmr's analysis).
Core distinction: These are two different investment targets, not two flavors of the same thing.
| Open Harness | Agent Framework / Runtime | |
|---|---|---|
| Invests in | Human-facing operation surface (使う面) | System-facing control substrate (組み込む面) |
| Primary user | Developer, operator, individual, small team | Product team, enterprise workflow team |
| Flexibility type | Broad — model switching, channel selection, prompt tweaking | Deep — typed state, graph transitions, tenant isolation, durable execution |
| Security model | Operator Safety — trusted human using agent safely | Product/Tenant Safety — untrusted users, multi-tenant isolation |
Two-axis evaluation framework (don't use a single "Security/Control/Ops" score):
Four lock-in types to analyze separately, not just "vendor lock-in":
Recommended architecture: 5-layer separation (Harness → Tool Boundary → Agent Control → State & Governance → Execution). Design principle: "Harnessを捨てても業務ロジックが残る" (business logic survives even if you abandon the harness).
Selection heuristic:
Key wiki pages for this topic:
[...existing content preserved...]
When comparing OpenClaw and Hermes Agent, the most productive conclusion is often not competitive — the two harnesses complement each other in a dual-agent architecture validated by community consensus (~20% of users in Kilo Reddit analysis, 1,300+ comments):
sessions_spawn({ runtime: "acp" })), cron/webhook scheduling, agent-to-agent communication tools.When to recommend this architecture: Complex multi-agent workflows, production deployments needing external validation (Hermes self-evaluation is unreliable — OpenClaw orchestration validates output quality), any setup where the user can afford running two processes.
Key wiki pages: [[comparisons/hermes-vs-openclaw-architecture]] (full evidence section with 8+ sources), [[entities/openclaw]] (§Orchestration Capabilities), [[entities/hermes-agent]] (§Execution Specialist Role).
When evaluating a model-harness pairing:
General compatibility tiers:
| Tier | Models | Compatible Harnesses | Notes |
|---|---|---|---|
| 🥇 | Claude Opus 4.7 / Sonnet 4.6 | Claude Code (native), Cursor (best score 93%), OpenCode, Aider | Best reasoning, but Anthropic wall on non-Anthropic harnesses |
| 🥇 | GPT-5.4 / GPT-5 | Codex CLI (native), Pi (no wall), OpenCode, Aider, Cursor | No subscription restrictions. App scaffolding specialist |
| 🥇 | Qwen 3.5 Coder 32B | Pi (local GGUF/MLX optimized), OpenCode, Aider | Best open-weight coder. Excellent price-performance |
| 🥇 | DeepSeek V3/V4 | OpenCode (75+ providers), Aider, Pi, OpenClaw (Ollama) | Cost-efficient. Strong at multi-file tasks |
| 🥈 | Gemma 4 26B | Pi + LM Studio (local), OpenCode (Ollama) | 18GB VRAM Q4_K_M; Pi's <1K system prompt compensates for weaker model |
| 🥈 | Gemini 2.5 Pro | Pi, OpenCode, Aider | Large free tier, good at long context |
| 🥉 | Local (7B-14B) | Pi only (minimal overhead), OpenCode (with performance tax) | Smaller models are overwhelmed by large system prompts |
Structure comparison responses with:
When an opinion leader (0xSero, Karpathy, Simon Willison, etc.) posts a ranking or assessment on X/Twitter:
web_extract on the tweet URL (e.g., https://x.com/handle/status/123456). This returns plain text of the tweet content. Do NOT use xurl for public tweet content — web_extract works directly.~/wiki/entities/. If not, create one via multi-source web research (GitHub, docs, site).## [Author]'s Assessment — #[Rank] for [Category] containing the opinion leader's exact quote and interpretation.concepts/agent-harness-comparison.md rather than creating a new page. Also update wiki/log.md and wiki/index.md.& characters. Double-quoted messages with & trigger the terminal tool's background-detection and fail.[[entities/<author>]] to the cross-references in log.md so the opinion leader's entity page connects to the contribution.comparisons/coding-agent-harnesses.md (May 1) and concepts/agent-harness-comparison.md (May 7) already exist. ALWAYS run session_search for 'agent harness comparison' before building new comparison artifacts. Duplicate comparison pages accumulate quickly.search_files in ~/wiki/entities/. The wiki may have multiple pages for the same person under slightly different filenames (e.g., hugo-bowne.md and hugo-bowne-anderson.md). Always merge into the older/more complete page.web_extract on tweet URLs, NOT xurl. xurl is for the user's own account operations (bookmarks, timeline). web_extract returns the plain text of public tweets reliably.& anywhere in the command string (even inside single-quoted strings) and interprets it as backgrounding. Do NOT use & in commit messages at all. Use and or + instead. A message like 'wiki: agent-harness & harness-commoditization' will fail — write 'wiki: agent-harness + harness-commoditization' instead.wiki/SCHEMA.md's canonical taxonomy. The git pre-commit hook rejects commits with unknown tags. Check SCHEMA.md before finalizing tags: grep -n "^-\\s*\\" wiki/SCHEMA.mdfor available categories. Common valid person-entity tags:person, agent-harness, ai-agents, coding-agents, open-source, entrepreneur`.web_extract. The first 5K+ characters are usually captured but the final ~800 chars may be lost. If the ending is truncated, check if the missing content is critical before re-extracting; browser_navigate is an alternative but requires Chrome installation.web_extract consistently truncates long articles (common with blog posts 8K+ chars), use execute_code with Python's httpx + BeautifulSoup as a fallback. Fetch with httpx.get(url, timeout=30), parse with BeautifulSoup(resp.text, 'html.parser'), extract the article element (soup.find('article') or soup.find(class_='md-content') for MkDocs sites), and print the text. This bypasses the LLM summarization timeout and returns the full raw text. Example pattern:
import httpx
from bs4 import BeautifulSoup
resp = httpx.get(url, timeout=30)
soup = BeautifulSoup(resp.text, 'html.parser')
article = soup.find('article') or soup.find('main') or soup.find(class_='md-content')
text = article.get_text(separator='\\n', strip=True) if article else soup.get_text(separator='\\n', strip=True)
print(text[:12000])