| name | observe-act-loop |
| description | Execute-observe-diagnose-fix loop for verifying code changes through execution. Covers local tests (pytest, unittest), local services, remote jobs (kubectl, submit), CI/CD pipelines, and cluster deployments. Use when code correctness requires runtime verification. Keywords: run tests, submit, deploy, verify, debug, resubmit, pytest, kubectl, check logs, watch, diagnose, tail logs. |
Observe-Act Loop
Unified methodology for the execute-observe-diagnose-fix cycle. All runtime verification follows this pattern regardless of scale.
When to Enter the Loop
Core question: does the change require execution to verify correctness?
Enter the loop when:
- Changed logic code with test coverage available
- Changed runtime behavior (process management, distributed config, server code)
- User reported an execution error, fix needs verification
- User explicitly requests: run, test, submit, verify, deploy, debug
Do NOT enter the loop when:
- Only docs, comments, type hints, or formatting changed
- Pure dead code deletion
- Change is obviously correct and user didn't ask to verify (e.g. fixing a typo)
Key: not every code change needs a loop. Entering the loop has cost (especially remote jobs). Ask yourself: how confident am I that this change is correct without executing it? If not confident enough, enter the loop.
Before Starting
Read the project's CLAUDE.md and any case-specific CLAUDE.md to find:
- Execute commands (test, submit, start)
- Observe methods (log paths, pod discovery, health checks)
- Success criteria (expected output, log lines, test pass counts)
If any of these are missing, ask the user before proceeding.
The Loop
Execute → Observe → Diagnose → Fix → Re-execute
↑ |
└──────────────────────────────────────┘
Step 1: Execute
Run the appropriate command for the verification level:
| Level | Example commands |
|---|
| Local test | pytest, pixi run -e dev pytest |
| Local service | Start server, curl endpoint |
| Remote job | pixi run submit examples/<name>/launch.sh |
| CI/CD | git push, trigger pipeline |
Step 2: Observe
Collect execution output. Where to look depends on the level:
| Level | Observation method |
|---|
| Local test | Read test output directly |
| Local service | Check stdout/stderr, probe endpoints |
| Remote job | kubectl get pods | grep <id>, kubectl exec <pod> -- tail -f <log> |
| CI/CD | gh run view, check pipeline UI |
For remote jobs: Follow the project's log location pattern. Typical: logs/<experiment_id>/node_<n>/<process>.log
Step 3: Diagnose
Follow these principles:
- Follow the signal chain. Check components in order, from entry point to failure.
- Find the boundary. What was the last successful step? What was the first failure?
- Don't skip ahead. No guessing. Read the actual error, trace the actual call path.
- Check the obvious first. Typos, wrong paths, missing env vars, import errors — before diving into logic bugs.
Common failure categories:
- Import/syntax error — immediate, fix and re-execute
- Configuration mismatch — wrong spec values, missing env vars
- Resource issue — OOM, GPU unavailable, network timeout
- Logic bug — wrong output, assertion failure, unexpected state
Step 4: Fix
Make the minimal change to address the diagnosed issue. Don't fix unrelated things in the same loop iteration.
Step 5: Re-execute
Go back to Step 1. Same command, same observation method.
Exit Condition
The loop ends when success criteria are met. Report to the user:
- What was executed
- What the outcome was
- Whether success criteria are satisfied
If success criteria aren't defined in the project CLAUDE.md, ask the user what "working" looks like before starting.
Verification Spectrum (Quick Reference)
Simplest: local test
run test → read output → fix → rerun
Medium: local service
start → probe → fix → restart
Heaviest: remote job
submit → find pod → background watch → fix on notify → kill old job → resubmit
Background watch: For remote jobs, prefer launching a background Agent (run_in_background=True) to watch logs instead of blocking the main conversation. Instruct the agent to report back immediately when it detects errors or anomalies.
Match the verification level to the change. Don't submit a remote job when a local test suffices.
Required Project Info Template
Projects should provide the following in their CLAUDE.md (fill only relevant sections):
## Verification
### Execute
- **Test command:** `<how to run tests>`
- **Submit command:** `<how to submit job/deploy>`
- **Start command:** `<how to start local service>`
### Observe
- **Test output:** `<where test results appear>`
- **Log location:** `<log path pattern>`
- **Process discovery:** `<how to find running processes/pods>`
- **Health check:** `<how to check if service is running>`
### Diagnose
- **Signal chain:** `<ordered list of components, check in order>`
- **Common failure patterns:** `<known error → likely cause mapping>`
### Success Criteria
- `<what "working" looks like>`
