Skip to main content

ctx-harness-evaluation

Run and interpret the CortexFS agent harness contract suite when validating protocol, authorization, ownership, cancellation or channel regressions. Use docs/evaluation.md for the separate live-model evaluation path.

소스 정보

저장소
LIghtJUNction/cortexfs
최근 소스 활동
2026년 9월 7일 06:09
감지된 SKILL.md 언어
영어
스타
4
포크
0

설치 방법

기본적으로 소스를 먼저 확인하는 Prompt가 선택됩니다. 직접 명령으로 전환하거나 로컬 사본을 다운로드할 수도 있습니다.

소스 파일 검토

설치 여부를 결정하기 전에 SKILL.md와 SkillsMP에 표시된 보조 파일을 읽어 보세요.

SKILL.md 표시 중

SKILL.md
소스 지침 · 읽기 전용 미리보기
name
ctx-harness-evaluation
description
Run and interpret the CortexFS agent harness contract suite when validating protocol, authorization, ownership, cancellation or channel regressions. Use docs/evaluation.md for the separate live-model evaluation path.
# CortexFS Harness Evaluation Read `AGENTS.md` and `docs/evaluation.md` from the repository root. The maintained runner is `evals/harness/run.py`; fixtures live beside the Rust modules and the required evidence is registered in `evals/harness/suites.json`. ```bash python3 evals/harness/run.py --list python3 evals/harness/run.py --suite context --suite wire python3 evals/harness/run.py --workspace --timeout 6600 ``` Choose focused groups for the changed contract; use `--workspace` for the full CI gate. The runner serializes Cargo itself. Do not start another Cargo process alongside it. A cold build is included in the timeout and elapsed time; periodic elapsed/log-size messages show progress without discarding raw evidence. Use a fresh output directory. Inspect `report.json`, `report.md` and raw logs. Missing or ignored required tests, zero tests, failures, timeout and interruption do not count as passing evidence. Report selected or unrun groups explicitly. Linux socket, identity and filesystem restrictions can prevent fixture execution; retain the failure and use a capable Linux environment rather than weakening a test. The default suite performs no model inference or paid provider calls. Passing contracts do not establish task accuracy, latency, token/cost or RSS improvements. For live-model requests, follow the existing provider-neutral setup and evidence requirements in `docs/evaluation.md` and `AGENTS.md`; use `cortexfs-performance` when making a performance change. Existing authorization remains authoritative. The retired Inspect/Pi runner is available only in Git history. Do not restore its scripts or duplicate an agent loop, JSONL parser or scoring engine here.
GitHub에서 보기