Skip to main content

model-toolkit-lm-evaluation

Task-based reasoning/knowledge evaluation through lm-eval.

Source facts

Repository
0xSero/model-toolkit
Last source activity
September 10, 2026 at 09:13
Detected SKILL.md language
English
Stars
4
Forks
1

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer
3 files

Showing SKILL.md

SKILL.md
Source instructions · Read-only preview
name
model-toolkit-lm-evaluation
description
Task-based reasoning/knowledge evaluation through lm-eval.
# lm-evaluation Use this tool when the user asks to task-based reasoning/knowledge evaluation through lm-eval. 1. Read this folder's [README](README.md) for prerequisites, supported inputs, commands, outputs and limits. 2. Resolve paths relative to the toolkit checkout. Run `python lm-evaluation/run.py --help` from its root. 3. Use explicit model/runtime/dataset identities, unique output paths, and environment variables for credentials. 4. For external GPU/container work, preview the command where `--dry-run` is supported and run the smallest useful smoke first. 5. Preserve request failures and raw evidence. State whether the result is a smoke, incomplete run, or complete evaluation. 6. A configuration, checksum, or startup pass does not establish model quality. Follow the README's downstream checks. Run this script from the user's installed environment; do not assume access to the original author's machines. This skill contains no background scheduling or automatic publishing instructions.
View on GitHub