name	arcagi3-attempt-review
description	Use when a remote ARC-AGI-3 run has finished and you need to diagnose what happened from scorecards, reasoning logs, and replay evidence

ARC-AGI-3 Attempt Review

Overview

Use this skill to review a completed run and decide the next most informative change.

Core principle: do not explain a bad score with one vague story when the artifacts can separate runner failures from reasoning failures.

When to Use

After a weak or surprising remote run
When comparing two attempts
When deciding whether to change prompts, policy, or runner code
When scorecards and replays exist but the failure mode is unclear

Do not use this skill before the run artifacts are available.

Review Order

Confirm the run provenance: game, scorecard ID, tags, config, and major hypothesis.
Inspect scorecard outcomes and any structured logs.
Inspect replay-worthy reasoning and notable action sequences.
Classify the failure.
Recommend one next change with a clear expected signal.

Failure Buckets

exploration failure - the agent never learned the key mechanic
planning failure - the agent learned pieces but did not sequence them well
execution failure - the chosen actions were poor despite having enough information
runner failure - resets, scorecards, action formatting, API handling, or orchestration were wrong

What Evidence To Use

scorecard fields and score deltas
replay links or recorded trajectories
reasoning metadata on important actions
available-action changes over time
whether actions were hypothesis-driven or blind

Recommendation Rules

Recommend the next most informative change, not the largest rewrite.
Distinguish between "needs better game understanding" and "needs better implementation."
Call out wasted actions explicitly.
If the run lacked enough metadata to diagnose well, say so and fix observability first.

Common Mistakes

blaming the policy when the runner was broken
using only final score instead of trajectory evidence
not separating exploration waste from necessary exploration
changing multiple variables before the next attempt

ARC-AGI-3 Attempt Review

Overview

Use this skill to review a completed run and decide the next most informative change.

Core principle: do not explain a bad score with one vague story when the artifacts can separate runner failures from reasoning failures.

When to Use

After a weak or surprising remote run

When comparing two attempts

When deciding whether to change prompts, policy, or runner code

When scorecards and replays exist but the failure mode is unclear

Do not use this skill before the run artifacts are available.

Review Order

Confirm the run provenance: game, scorecard ID, tags, config, and major hypothesis.

Inspect scorecard outcomes and any structured logs.

Inspect replay-worthy reasoning and notable action sequences.

Classify the failure.

Recommend one next change with a clear expected signal.

Failure Buckets

exploration failure - the agent never learned the key mechanic

planning failure - the agent learned pieces but did not sequence them well

execution failure - the chosen actions were poor despite having enough information

runner failure - resets, scorecards, action formatting, API handling, or orchestration were wrong

What Evidence To Use

scorecard fields and score deltas

replay links or recorded trajectories

reasoning metadata on important actions

available-action changes over time

whether actions were hypothesis-driven or blind

Recommendation Rules

Recommend the next most informative change, not the largest rewrite.

Distinguish between "needs better game understanding" and "needs better implementation."

Call out wasted actions explicitly.

If the run lacked enough metadata to diagnose well, say so and fix observability first.

Common Mistakes

blaming the policy when the runner was broken

using only final score instead of trajectory evidence

not separating exploration waste from necessary exploration

changing multiple variables before the next attempt

arcagi3-attempt-review