원클릭으로
quoracle
quoracle에는 shelvick에서 수집한 skills 4개가 있으며, 저장소 수준 직업 범위와 사이트 내 skill 상세 페이지를 제공합니다.
이 저장소의 skills
Coordinates LiveBench benchmark runs. Reads a pre-built manifest, dispatches one solver per question by sequential index using batch_async, then scores all answers via score-run.sh and produces the benchmark report. Use when running LiveBench evaluations. Do NOT use for MMLU-Pro or general tasks.
Coordinates MMLU-Pro benchmark runs. Reads a pre-built manifest, dispatches one answerer per question by sequential index using batch_async, then scores all answers via score-run.sh and produces the benchmark report. Use when running MMLU-Pro evaluations. Do NOT use for general Q&A tasks.
Solves a single LiveBench benchmark question. Reads the question from a file and answers it directly. Use when spawned by livebench-coordinator. Do NOT use for MMLU-Pro or non-benchmark tasks.
Answers a single MMLU-Pro multiple-choice question. Reads the question from a file, follows the prompt exactly, and writes the answer to a file. Use when spawned by mmlu-coordinator during benchmark runs. Do NOT use for general Q&A or non-benchmark tasks.