terminal-agent-benchmark

Run and maintain Con's terminal-agent benchmark against a live app session. Use when validating con-cli, SSH workspace reuse, tmux awareness, agent-target preparation, or when collecting benchmark evidence for regressions and release notes.

Ir para a instalação

Repositório: nowledge-co/con-terminal
Última atividade na origem: 30 de abril de 2026 às 15:25
Idioma detectado do SKILL.md: inglês
Estrelas: 556
Forks: 44

Opções de instalação

Por padrão, está selecionado o prompt que primeiro revisa a origem. Você pode mudar para um comando direto ou baixar uma cópia local.

Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.

Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.

npx skills add https://github.com/nowledge-co/con-terminal --skill terminal-agent-benchmark

O comando permanece em uma só linha. Role horizontalmente para revisá-lo antes de copiar.

Prefere uma cópia local? Baixe os arquivos disponíveis atualmente no SkillsMP.

Revise os arquivos de origem

Leia o SKILL.md e os arquivos complementares exibidos pelo SkillsMP antes de decidir se vai instalar.

SKILL.md

Instruções da origem · Visualização somente leitura

Voltar à pesquisa Voltar à instalação

Terminal Agent Benchmark

Use this skill when you need to evaluate Con as a terminal-native agent, not just compile it.

Primary references:

Default workflow

Confirm a live app session and socket exist.

Run the strict benchmark:

python3 benchmarks/terminal-agent/run.py --suite strict

If provider setup is present, run:

CON_BENCH_ENABLE_AGENT=1 python3 benchmarks/terminal-agent/run.py --suite all

Prefer a built-in profile when one matches the workflow:

python3 benchmarks/terminal-agent/run.py --list-profiles
python3 benchmarks/terminal-agent/run.py --profile basic-local-shell
CON_BENCH_ENABLE_AGENT=1 python3 benchmarks/terminal-agent/run.py --profile basic-local-codex --suite all

Use starter profiles for quick regression checks and operator profiles for richer coding, SSH maintenance, or tmux dev-loop evaluation.

python3 benchmarks/terminal-agent/run.py --profile operator-local-codex-devloop --suite operator
python3 benchmarks/terminal-agent/run.py --profile operator-local-claude-devloop --suite operator
python3 benchmarks/terminal-agent/run.py --profile operator-local-opencode-devloop --suite operator
python3 benchmarks/terminal-agent/run.py --profile operator-ssh-dual-host-maintenance --suite operator
python3 benchmarks/terminal-agent/run.py --profile operator-ssh-tmux-devloop --suite operator

For SSH/tmux changes, run the relevant playbook under benchmarks/terminal-agent/playbooks/.

Save the JSON record under .context/benchmarks/ and cite it in your report.

If the run is an operator benchmark, score it with:

python3 benchmarks/terminal-agent/score.py --profile ... --record ... --score ...

Generate a trend report when comparing many runs:

python3 benchmarks/terminal-agent/report.py

Rules

Prefer pane_id over pane_index when following up on benchmark findings.

Keep benchmark control on the existing panes.* API unless the benchmark explicitly targets pane-local surfaces. Surface support is additive and should not change the built-in agent's pane/tool contract.

Do not treat playbook observations as strict pass/fail evidence unless the behavior is actually deterministic.

If a scenario depends on host setup, say so explicitly.

Keep operator playbooks safe-by-default. Prefer read-only checks first, and treat destructive or privileged steps as explicit branches.

If a benchmark reveals a product limit, document the limit instead of hiding it behind a softer assertion.

Operator suites intentionally serialize agent ask turns. If a tab already has a pending agent request, let the runner wait and reuse the same tab instead of opening a parallel benchmark against it.