| name | debug-challenge |
| description | Debugs an existing challenge by starting it, inspecting tasks, troubleshooting over SSH, fixing source files, and restarting. Use when a challenge's tasks are failing or hanging. |
| argument-hint | <challenge-name> |
| disable-model-invocation | true |
Debug the challenge $0. Follow these steps in order.
Debugging why a solution doesn't solve? Prefer the CI helper path (the
solve-challenge skill / tests/challenges/main.go --skip-auth) instead of the manual
flow below. It starts its own playground and, while it runs, you can point ordinary
labctl commands at that playground (labctl playground tasks <id> -o yaml,
labctl ssh <id> -- <diag>). Most importantly it writes a per-solution artifacts
directory (path printed at the start of the run) whose files are appended dynamically
during the attempt — chunk-NN.*.log (per-chunk output), tasks-*.txt
(labctl playground tasks -o yaml snapshots every 15s), and journal-<machine>.examiner.log.
Analyzing those artifacts is the fastest way to find the failing chunk or task. The manual
flow below is for debugging the challenge environment/tasks directly.
1. Start the challenge in the background
Run the challenge start command in the background (it may block waiting for init tasks):
labctl challenge start $0 --no-open --no-ssh
Do not wait for it to complete - proceed immediately to the next step.
2. Find the playground ID
Use the list-running-playgrounds skill (or labctl playground list) to find the
playground ID corresponding to this challenge. You'll need this ID for all subsequent steps.
3. Inspect examiner tasks
Get the overview via the API (no SSH tunnel) - poll this in a loop while init
tasks are still running:
labctl playground tasks <playID> -o yaml
Tasks are edge-activated and never regress: once a task shows completed
(40) it stays there for the life of the playground, so a single completed
observation is authoritative.
SSH discipline: run labctl ssh calls foreground and one at a time - never in a
backgrounded retry loop. Orphaned labctl ssh children exhaust the tunnel pool and
every later ssh then fails with StartTunnel(): retry after 2562047h47m16s. If you
hit that, pkill -9 -f 'labctl ssh' and retry (see run-playground-command).
4. Troubleshoot over SSH
If the task output is not enough to identify the problem, run commands on the
playground VM using the run-playground-command skill:
labctl ssh <playID> -- <diagnostic-command>
Common diagnostics: checking file contents, running scripts manually, inspecting
service status, reviewing logs, testing network connectivity, etc.
Tip: If the machine you need to inspect has noSSH: true, you can still read its
task status and last-run output via labctl playground tasks <playID> -o yaml (the API
covers hidden machines too). Only when you need to run live diagnostic commands on
that machine must you temporarily comment out the directive (#noSSH: true), push, and
restart the challenge. Remember to restore it when done debugging.
5. Fix the challenge source
Based on your findings, edit the challenge's local source files (typically index.md)
to fix the failing tasks. Common fixes include:
- Adjusting init task scripts
- Fixing verification task conditions
- Correcting expected outputs or paths
6. Push changes and restart the challenge
Important: After pushing changes, you must restart the challenge for the changes
to take effect. The running playground uses the old version of the content.
First, push the changes using the edit-remote-content skill:
labctl content push challenge $0 --dir <folder> --force
Then stop the current attempt and start a fresh one (the stop is required - a
running attempt keeps serving the old content):
labctl challenge stop $0
labctl challenge start $0 --no-open --no-ssh
Go back to step 2 and verify that the tasks now pass.
7. Repeat if needed
If tasks are still failing, repeat steps 2-6 until all tasks pass.
