菜单
Launch the vLLM servers needed for DeepHone experiments — the gpt-oss search agent (with tool calling), the reranker (gpt-oss listwise or Qwen3-Reranker-0.6B cross-encoder), and the gpt-oss-120b LLM-as-judge. Use before running or evaluating deep-research runs.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/sahel-sh/DeepHone --skill serve-vllm-models命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Evaluate a DeepHone deep-research run with the gpt-oss-120b LLM-as-judge (accuracy / recall / calibration), aggregate token usage, and compute the Effective Token Cost (ETC) plus the paper's accuracy-vs-cost figures (Figures 1-3). Use after producing run directories with run-reranking-experiment.
Add a custom retriever (searcher) or reranker to DeepHone by implementing the BaseSearcher / BaseReranker interface and registering it in the SearcherType / RerankerType enum. Use when integrating a new retrieval or reranking method into the benchmark.
Run the paper's core experiments — one-shot reranking effectiveness (Table 1) and end-to-end deep-research with listwise or cross-encoder reranking at depth d in {10,20,50} and search reasoning in {low,medium,high} (Tables 2/4, Figures 2/3). Use to produce run directories over BrowseComp-Plus.
基于 SOC 职业分类
正在显示 SKILL.md
| name | serve-vllm-models |
| description | Launch the vLLM servers needed for DeepHone experiments — the gpt-oss search agent (with tool calling), the reranker (gpt-oss listwise or Qwen3-Reranker-0.6B cross-encoder), and the gpt-oss-120b LLM-as-judge. Use before running or evaluating deep-research runs. |
Deep-research with reranking needs two model servers (reranker + search agent), each on its own GPU; evaluation needs a judge server. Start each in the background / a separate terminal.
Listwise (paper default):
CUDA_VISIBLE_DEVICES=0 vllm serve openai/gpt-oss-20b --port 44713 \
--dtype auto --gpu-memory-utilization 0.8 --enable-prefix-caching --max-model-len 32768
Cross-encoder (heterogeneous setup) — serve the reranker instead as:
CUDA_VISIBLE_DEVICES=0 vllm serve Qwen/Qwen3-Reranker-0.6B --port 44713 --dtype auto
CUDA_VISIBLE_DEVICES=1 vllm serve openai/gpt-oss-20b --port 34713 \
--dtype auto --gpu-memory-utilization 0.8 --tool-call-parser openai --enable-auto-tool-choice
Use openai/gpt-oss-120b (and --tensor-parallel-size N) for the larger agent.
CUDA_VISIBLE_DEVICES=0 vllm serve openai/gpt-oss-120b --port 21292 --dtype auto --gpu-memory-utilization 0.8
Poll /v1/models before launching a run:
until curl -sf "http://localhost:44713/v1/models" | grep -q '"id"'; do sleep 10; done
Ports are arbitrary but must match the --reranker-base-url, --model-url, and --base_url flags used in
run-reranking-experiment and evaluate-and-etc. For OpenAI/Anthropic/Gemini/etc. agents (no local
serving), see the guides in docs/.