القائمة
Judge model training skill for TTC evaluation. Use when training models to evaluate and score outputs from other models. Supports reward model training and judge datasets.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Transform HuggingFace/arXiv research papers into trainable engineering features, skills, scripts and configs for Qwen 3.5 0.8B. Use when user provides a paper URL or wants to convert research into implementation. Creates complete skill ecosystems with ASCII diagrams.
TTC distillation skill for knowledge transfer from teacher to student models. Use when distilling large models to smaller ones with TTC awareness. Supports medium and high profiles.
Hyperparameter tuning skill for TTC training using Optuna. Use when optimizing learning rate, batch size, LoRA rank, and other training parameters.
TTC router skill for inference scaling decisions. Use when training, evaluating, or benchmarking the router that decides between low/medium/high/superthink compute modes.
TTC-aware training skill for Qwen models. Use when training models with Test-Time Compute awareness, including CPT → SFT → RLVR → TTC pipeline stages. Supports Kaggle and HF Jobs backends.
| name | ttc-judge |
| description | Judge model training skill for TTC evaluation. Use when training models to evaluate and score outputs from other models. Supports reward model training and judge datasets. |
| compatibility | Requires judge training dataset and compute backend |
Train judge models for evaluating model outputs in the TTC stack.
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Query │ │ Response │ │ Judge │
│ │────▶│ Model │────▶│ Model │
└─────────────┘ └─────────────┘ └──────┬──────┘
│
▼
┌─────────────┐
│ Score │
│ 0.0 - 1.0 │
└─────────────┘
{"query": "...", "response": "...", "score": 0.9, "rationale": "Correct and well-explained"}
{"query": "...", "response": "...", "score": 0.3, "rationale": "Partially correct but incomplete"}
python scripts/train_judge.py \
--base-model Qwen/Qwen2.5-0.5B \
--dataset path/to/judge_data.jsonl \
--output models/ttc-judge \
--backend kaggle
| Type | Purpose | Output |
|---|---|---|
| Scorer | Rate response quality | 0.0-1.0 |
| Ranker | Compare two responses | A or B |
| Verifier | Check correctness | True/False |
| Critic | Provide feedback | Text |
python scripts/eval_judge.py \
--judge models/ttc-judge \
--benchmark judge-bench \
--metrics accuracy,correlation
from ttc_judge import JudgeModel
judge = JudgeModel.from_pretrained("models/ttc-judge")
# Score responses
for response in responses:
score = judge.score(query, response)
if score > 0.8:
accept(response)
User: Train a judge to evaluate math solutions
Agent will:
1. Prepare judge dataset with scored examples
2. Train judge model
3. Evaluate on held-out set
4. Return judge model path