findalexli

Create a benchmark task from a GitHub PR. Use when generating new harbor tasks from merged PRs — fetches PR metadata, discovers agent configs, generates tests, Dockerfile, solve.sh, and eval_manifest.yaml.

Validate a harbor task with Docker oracle test — builds the image, runs nop test (expect reward=0), applies gold patch, runs gold test (expect reward=1). Use after scaffolding to verify task quality.

Find candidate GitHub PRs for new benchmark tasks. Searches repos for merged PRs with agent config files, filters by quality heuristics, and outputs JSONL candidates. Use when expanding the task pool.

Search for academic papers using Semantic Scholar, OpenReview, OpenAlex, and CrossRef. Use when you need to find related work, check novelty, gather citations, download/read full papers, or get BibTeX.