メニュー
Use when analyzing RL training status from W&B or local logged metrics, especially actor KL loss, PPO KL, grad norm, clipfrac, critic score/reward/advantages, response length, global sequence length, entropy, invalid draft rates, and diagnosing failures by checking run config and Search-R1/verl implementation.
Use when you want to commit and push recent changes to GitHub. Automatically summarizes what changed into a short commit message, then runs git add ., git commit, and git push. Triggers: commit, push, sync to github, save changes, git commit and push.
Use when summarizing a session and updating PROJECT_STATUS.md with important changes, decisions, active experiment state, new constraints, or new files. Triggers: update project status, summarize session, handoff, record changes, keep PROJECT_STATUS current, end of session.
| name | rl-training-diagnoser |
| description | Use when analyzing RL training status from W&B or local logged metrics, especially actor KL loss, PPO KL, grad norm, clipfrac, critic score/reward/advantages, response length, global sequence length, entropy, invalid draft rates, and diagnosing failures by checking run config and Search-R1/verl implementation. |
Use this skill to diagnose whether a Search-R1/verl RL run is healthy, unstable, reward-hacking, or misconfigured.
Locate the run artifacts.
monitor_outputs/rl_metrics_parsed.csv, recent *.log, and wandb/run-*/files/output.log.Generate a metrics summary.
python skills/rl-training-diagnoser/scripts/summarize_rl_metrics.py --metrics-csv <csv> --log <optional-log> --format markdown.--format json when a structured intermediate is easier to reason over.Inspect training configuration when needed.
use_kl_loss, actor.kl_loss_coef, algorithm.kl_ctrl.kl_coef, adv_estimator, actor.state_masking, learning rate, batch size, rollout count, loss aggregation, and reward weights.Connect metrics to implementation.
references/search-r1-rl-diagnostics.md for project-specific mappings.Report with this structure.
verl/trainer/main_ppo.py for paper-writing reward and logged custom metrics.verl/trainer/ppo/ray_trainer.py for KL penalty application and reward path.verl/trainer/ppo/core_algos.py for GRPO advantage aggregation.verl/workers/actor/dp_actor.py for actor loss, KL loss, and masking.search_r1/llm_agent/generation.py for rollout loop, inserted instructions, and info_mask.train_paper_writing_*.sh for default run parameters.