メニュー
Parse classic SWE-agent .traj trajectories: one step per entry of the trajectory array, in order, matching the CodeTraceBench annotation convention. No upstream CodeTracer commit ships a SWE-agent parser.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/tangle-network/agent-eval --skill sweagent-trajコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
Maintain agent-eval cases, judges, records, traces, campaigns, comparisons, and releases.
Parse OpenHands trajectories published as per-call LiteLLM completion logs (<model>-<timestamp>.json files with messages/response/kwargs/cost), the layout of CodeTraceBench's swe_raw OpenHands trials. Reads the call file whose messages hold the most tool-call steps (history condensation can truncate later views) and emits one step per non-finish assistant tool call.
Parse OpenHands session-event trajectories at the CodeTraceBench annotation granularity: one step per action event with a cause-paired observation (run, run_ipython, read, edit, think, recall). Outranks the seed openhands skill, which counts only run/run_ipython and undercounts.
SOC 職業分類に基づく
SKILL.md を表示中
| name | sweagent_traj |
| description | Parse classic SWE-agent .traj trajectories: one step per entry of the trajectory array, in order, matching the CodeTraceBench annotation convention. No upstream CodeTracer commit ships a SWE-agent parser. |
| fingerprints | ["*.traj"] |
| priority | 60 |
| metadata | {"version":"1.0","source":"agent-eval codetracebench-oht2-20260801"} |
trial_dir/
<trial>.traj # JSON: environment, trajectory[], history[], info, context
<trial>.context.json # edited/viewed file context (not steps)
<trial>.patch # final submission patch
report.json # evaluation outcome
The run directory must hold exactly one *.traj file; zero or several
refuse detection instead of guessing.
trajectory[] entry, in array order. Every entry counts,
including entries whose action is blank.action; actions that are blank after
trimming render as their JSON literal so the step survives non-empty
action gates losslessly.observation passes through unchanged (empty strings stay
empty strings); a non-string, non-null observation is a parse error.thought becomes thinking when it is a non-blank string.action_ref points at <trial>.traj#trajectory[i] and embeds the raw
entry JSON (action, observation, response, state, thought), so the full
source record stays recoverable per step; observation_ref stays null
because it would duplicate the same entry.history[] message that is not
the interface demonstration (marker: --- DEMONSTRATION ---). Both
published prompt templates (<pr_description> upload and ISSUE: text)
sit in that message on all 108 verified rows.This enumeration reproduces the CodeTraceBench annotation step_count on
106 of 108 verified SWE-agent rows at dataset revision
aa213b84ffb6690fc37ca15766d6ca174ec36d4d. The other 2 rows
(keras-team__keras-19775, mui__material-ui-11451) publish a
function-call-style trajectory (45/48 and 92/94 entries with empty
response) whose annotated view condensed the run (26 vs 48, 54 vs 94
entries); no published view reproduces those counts, so they fail the
step-count gate loudly instead of importing misaligned labels.