ワンクリックで
quoracle
quoracle には shelvick から収集した 4 個の skills があり、リポジトリ単位の職業カバレッジとサイト内 skill 詳細ページを表示します。
このリポジトリの skills
Coordinates LiveBench benchmark runs. Reads a pre-built manifest, dispatches one solver per question by sequential index using batch_async, then scores all answers via score-run.sh and produces the benchmark report. Use when running LiveBench evaluations. Do NOT use for MMLU-Pro or general tasks.
Coordinates MMLU-Pro benchmark runs. Reads a pre-built manifest, dispatches one answerer per question by sequential index using batch_async, then scores all answers via score-run.sh and produces the benchmark report. Use when running MMLU-Pro evaluations. Do NOT use for general Q&A tasks.
Solves a single LiveBench benchmark question. Reads the question from a file and answers it directly. Use when spawned by livebench-coordinator. Do NOT use for MMLU-Pro or non-benchmark tasks.
Answers a single MMLU-Pro multiple-choice question. Reads the question from a file, follows the prompt exactly, and writes the answer to a file. Use when spawned by mmlu-coordinator during benchmark runs. Do NOT use for general Q&A or non-benchmark tasks.