菜单
Auto-label frames with bounding boxes. Supports four modes — CUA+SAM (best accuracy, OpenAI CUA clicks + SAM segmentation), Gemini (native bbox detection), GPT vision (API fallback), or Codex vision subagents (no API keys). Parallel dispatch via git worktrees. Use after collecting frames.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
Download YouTube videos and extract frames using yt-dlp and ffmpeg for YOLO training data. Use when you need to gather frames from a video URL.
Evaluate trained YOLO model accuracy with mAP metrics and per-class error analysis. Identifies failure cases for targeted improvement. Use after training.
Train a custom YOLO object detection model from any YouTube gameplay video. Provide a video URL and target classes, and this skill handles the entire pipeline autonomously — frame extraction, AI-powered labeling, data augmentation, training, and evaluation with iterative improvement.
Generate synthetic training data variations using image transforms. Increases dataset diversity with flips, brightness jitter, and noise. Use after labeling.
Train a YOLO model on labeled dataset using ultralytics. Splits data into train/val, generates dataset.yaml, and produces trained weights. Use after labeling and augmenting.
基于 SOC 职业分类
| name | label |
| description | Auto-label frames with bounding boxes. Supports four modes — CUA+SAM (best accuracy, OpenAI CUA clicks + SAM segmentation), Gemini (native bbox detection), GPT vision (API fallback), or Codex vision subagents (no API keys). Parallel dispatch via git worktrees. Use after collecting frames. |
Set label_mode in config.json:
| Mode | How it works | Best for |
|---|---|---|
cua+sam | CUA clicks on objects → SAM segments precise boundaries | Best accuracy, hackathon demo |
gemini | Gemini native bounding box detection (0-1000 scale) | Fast, good native bbox support |
gpt | GPT vision model returns JSON bounding boxes | Simple fallback |
codex | Codex subagents view images and write YOLO labels directly | No API keys |
Read config.json for label_mode, classes, model, num_agents
If the user asks to call subagent, route to parallel dispatch in step 5.
CUA+SAM mode (recommended):
Run: uv run .agents/skills/label/scripts/label_cua_sam.py
Requires: OPENAI_API_KEY, classes must be set in config.json
Gemini mode:
Run: uv run .agents/skills/label/scripts/label_gemini.py
Requires: GEMINI_API_KEY or GOOGLE_API_KEY
GPT mode (fallback):
Run: uv run .agents/skills/label/scripts/run.py
Requires: OPENAI_API_KEY
Parallel dispatch (GPT or Codex mode):
Run: bash .agents/skills/label/scripts/dispatch.sh [num_agents]
Creates N git worktrees, dispatches N Codex subagents, merges results.
If Codex subagents are unavailable in-session, this shell command is the fallback path.
Supports:
label_mode=gpt with OPENAI_API_KEY (runs run_batch.py)label_mode=codex without API keys (Codex image-viewing subagents)Outputs: output/frames/*.txt (YOLO labels), output/classes.txt
| Script | Mode | Description |
|---|---|---|
label_cua_sam.py | cua+sam | CUA for clicks + SAM for segmentation |
label_gemini.py | gemini | Gemini native bounding boxes |
run.py | gpt | GPT vision structured output |
run_batch.py | gpt | GPT vision (subagent batch mode) |
dispatch.sh | gpt/codex | Parallel subagent orchestrator |
merge_classes.py | all | Unify class maps from subagents |
auto_label_and_show.py | all | Auto-run configured labeler and print/render label previews |