メニュー
Judge model training skill for TTC evaluation. Use when training models to evaluate and score outputs from other models. Supports reward model training and judge datasets.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Transform HuggingFace/arXiv research papers into trainable engineering features, skills, scripts and configs for Qwen 3.5 0.8B. Use when user provides a paper URL or wants to convert research into implementation. Creates complete skill ecosystems with ASCII diagrams.
TTC distillation skill for knowledge transfer from teacher to student models. Use when distilling large models to smaller ones with TTC awareness. Supports medium and high profiles.
Hyperparameter tuning skill for TTC training using Optuna. Use when optimizing learning rate, batch size, LoRA rank, and other training parameters.
TTC router skill for inference scaling decisions. Use when training, evaluating, or benchmarking the router that decides between low/medium/high/superthink compute modes.
TTC-aware training skill for Qwen models. Use when training models with Test-Time Compute awareness, including CPT → SFT → RLVR → TTC pipeline stages. Supports Kaggle and HF Jobs backends.
| name | ttc-judge |
| description | Judge model training skill for TTC evaluation. Use when training models to evaluate and score outputs from other models. Supports reward model training and judge datasets. |
| compatibility | Requires judge training dataset and compute backend |
Train judge models for evaluating model outputs in the TTC stack.
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Query │ │ Response │ │ Judge │
│ │────▶│ Model │────▶│ Model │
└─────────────┘ └─────────────┘ └──────┬──────┘
│
▼
┌─────────────┐
│ Score │
│ 0.0 - 1.0 │
└─────────────┘
{"query": "...", "response": "...", "score": 0.9, "rationale": "Correct and well-explained"}
{"query": "...", "response": "...", "score": 0.3, "rationale": "Partially correct but incomplete"}
python scripts/train_judge.py \
--base-model Qwen/Qwen2.5-0.5B \
--dataset path/to/judge_data.jsonl \
--output models/ttc-judge \
--backend kaggle
| Type | Purpose | Output |
|---|---|---|
| Scorer | Rate response quality | 0.0-1.0 |
| Ranker | Compare two responses | A or B |
| Verifier | Check correctness | True/False |
| Critic | Provide feedback | Text |
python scripts/eval_judge.py \
--judge models/ttc-judge \
--benchmark judge-bench \
--metrics accuracy,correlation
from ttc_judge import JudgeModel
judge = JudgeModel.from_pretrained("models/ttc-judge")
# Score responses
for response in responses:
score = judge.score(query, response)
if score > 0.8:
accept(response)
User: Train a judge to evaluate math solutions
Agent will:
1. Prepare judge dataset with scored examples
2. Train judge model
3. Evaluate on held-out set
4. Return judge model path