메뉴
Plan and interpret model evaluations and ablations for post-training research. Use when comparing checkpoints, designing ablations, selecting benchmarks, or summarizing what changed after training.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
Run LLM post-training on Tinker with CPU-side orchestration and remote GPU execution. Use when preparing or launching SFT, DPO, or PPO-style runs, checkpointing, sampling checkpoints, or resuming long-running jobs through Hermes Research Agent.
Run durable end-to-end LLM post-training research loops from zero-spec ideas to finished reports. Use when the goal is autonomous literature review, hypothesis selection, experiment planning, Tinker training, evaluation, checkpointing, and resumable long-running research work.
Convert literature findings into concrete LLM experiment plans. Use when the agent has papers, blog posts, or benchmark notes and needs to turn them into hypotheses, training plans, dataset choices, and evaluation criteria.
Generate and score candidate LLM research ideas from sparse user goals. Use when a project starts with no concrete benchmark, dataset, or training recipe and the agent needs to propose plausible, testable directions.
Produce clear iteration memos and final research summaries from project artifacts. Use when converting experiment state, training outcomes, evaluation results, and open questions into reports for later review.
Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support
| name | eval-and-ablation |
| description | Plan and interpret model evaluations and ablations for post-training research. Use when comparing checkpoints, designing ablations, selecting benchmarks, or summarizing what changed after training. |
| version | 1.0.0 |
| author | Hermes Research Agent |
| license | MIT |
| metadata | {"hermes":{"tags":["evaluation","ablation","benchmarking","analysis"]}} |
