Menu
Runnable evaluation template scripts for ML tasks. Match task_type to template, adapt CONFIG, run.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.
Create and manage datasets on Hugging Face Hub. Supports initializing repos, defining configs/system prompts, streaming row updates, and SQL-based dataset querying/transformation. Designed to work alongside HF MCP server for comprehensive dataset workflows.
Add and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.
Based on SOC occupation classification
| name | eval-templates |
| description | Runnable evaluation template scripts for ML tasks. Match task_type to template, adapt CONFIG, run. |
Match the plan's task_type to a template in scripts/, adapt CONFIG variables, run via run_code_in_docker.
| task_type | Script | Notes |
|---|---|---|
| extractive_qa | eval_qa.py | Pipeline + no-answer threshold for SQuAD v2 |
| nli, classification | eval_nli.py | Label remapping, filter label=-1 |
| zero_shot | eval_zero_shot.py | Pipeline + candidate_labels |
| multilabel | eval_multilabel.py | Single-label argmax for accuracy metric |
| summarization | eval_summarization.py | Seq2Seq + ROUGE |
| ner | eval_ner.py | Token classification + seqeval, parquet fallback |
| vision | eval_vision.py | AutoImageProcessor, grayscale→RGB |
| sentiment | eval_sentiment.py | Standard classification |
| multiple_choice | eval_multiple_choice.py | vLLM + conditional log-likelihood |
cat scripts/eval_<type>.py