paper-liweiwu-2026-ai-co-scientist-ranking-search-models

Stars0

Forks0

UpdatedMay 20, 2026 at 04:18

Use when the agent is positioning the consuming agent/manuscript against domain-specific-ai-scientist literature, borrowing a agent-runtime-full-pipeline-cloud design pattern, or auditing a benchmark/manuscript claim related to it - central claim is "An AI Co-Scientist framework automates the full search-ranking research pipeline (idea -> code -> GPU training -> analysis) with single-LLM agents for routine t" (arXiv preprint 2026).

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Source

huangzesen

huangzesen/heliosi-501-corpus-skill

View GitHub Repository View Creator Repositories

Download

Related occupationsSOC

Based on SOC occupation classification

PhysicistsLife, Physical, and Social Science Occupations·SOC 19-2012

File Explorer

2 files

SKILL.md

readonly

More from this repository

same repository

heliosi-501-corpus

huangzesen/heliosi-501-corpus-skill

Use when the user asks about HelioSI, the heliophysics paper-skill corpus, the 501-skill corpus, paper-to-skill compilation, generating hypotheses or experiments from heliophysics literature, selecting corpus entries by topic (PFSS, open flux, PSP switchbacks, SEPs, turbulence, solar-wind classification, instruments, MHD, agent-runtime design), or deciding which papers to verify against full text.

2026-05-210

hu-2022-deep-swim-cnn-discontinuities

huangzesen/heliosi-501-corpus-skill

Use when running Deep-SWIM, a semi-supervised CNN pipeline that classifies 5-minute Wind/MFI magnetic-field windows (3s-resampled, B_x/B_y/B_z stacked as channels) as discontinuity vs ambient solar wind. Central paper claim: a four-layer 1D-CNN trained with contrastive learning + pseudo-labeling + Online Hard Example Mining (OHEM) reaches AUC ≈ 0.90 on the held-out validation set and precision 0.52 / recall 0.73 / AUC 0.82 on a one-day expert-hand-labelled test set (2018-11-18), under heavy class imbalance (~15 % discontinuity-positive windows). NeurIPS 2021 Machine Learning and the Physical Sciences workshop (Lamdouar et al. 2022, arXiv:2203.01184). Note: the slug retains a legacy first-author placeholder; the verified lead author is Hala Lamdouar (University of Oxford), not Hu.

2026-05-200

katsavrias-2025-periodic-density-structures-solar-orbiter

huangzesen/heliosi-501-corpus-skill

Use when detecting and cataloguing quasi-Periodic Density Structures (PDSs) in Solar Orbiter density data with Multitaper + wavelet, and quantifying their radial-length-scale evolution between 0.3 and 1 au — central claim is slow-wind PDSs expand at ~10% and fast-wind PDSs compress at ~10% across that range (Katsavrias et al. 2025, arXiv:2511.15518; venue TODO verify).

2026-05-200

sasli-2026-ember-modulated-ion-acoustic-wave-ml

huangzesen/heliosi-501-corpus-skill

Use when running EMBER (Electron heating from Modulated Burst-mode Event Recognition) on PSP FIELDS Digital Burst Memory (DBM) voltage waveforms to detect modulated ion-acoustic waves (mIAWs — including triggered IAWs and frequency-dispersed IAWs) and correlate them with anomalous core-electron heating. Central paper claim: an ensemble of 16 background-only anomaly detectors (physics-motivated + classical outlier + deep-learning), run on log-scaled Fourier spectrograms of DBM bursts from PSP Encounters 6–9 and 15, recovers ≈ 93 % of labelled mIAW events at an estimated 1 % FAR on a curated catalog of 42 anomalous + 496 background spectrograms; flagged intervals show core perpendicular T_e above adiabatic cooling and elevated T_e/T_i ratios, reproducing previously reported heating phenomenology without using temperatures in the detection step. Sasli et al. 2026, arXiv:2605.00162, submitted to Earth and Space Science.

2026-05-200

paper-gottweis-2025-ai-co-scientist-multi-agent-gemini

huangzesen/heliosi-501-corpus-skill

Use when the agent is positioning the consuming agent/manuscript against ai-scientist-system literature, borrowing a agent-runtime-async-tournament design pattern, or auditing a benchmark/manuscript claim related to it - central claim is "A multi-agent system on Gemini 2.0 implementing a generate-debate-evolve loop with asynchronous task execution and tournament-style hypothesis evolution improve" (arXiv preprint 2025).

2026-05-200

paper-llmtreesearch-2026-multi-pathogen-disease-forecasting

huangzesen/heliosi-501-corpus-skill

Use when the agent is positioning the consuming agent/manuscript against domain-specific-ai-scientist literature, borrowing a agent-runtime-tree-search-executable-software design pattern, or auditing a benchmark/manuscript claim related to it - central claim is "An autonomous LLM-guided tree-search system iteratively generates, evaluates, and optimizes executable forecasting software for multi-pathogen disease forecasti" (arXiv preprint 2026).

2026-05-200

name	paper-liweiwu-2026-ai-co-scientist-ranking-search-models
description	Use when the agent is positioning the consuming agent/manuscript against domain-specific-ai-scientist literature, borrowing a agent-runtime-full-pipeline-cloud design pattern, or auditing a benchmark/manuscript claim related to it - central claim is "An AI Co-Scientist framework automates the full search-ranking research pipeline (idea -> code -> GPU training -> analysis) with single-LLM agents for routine t" (arXiv preprint 2026).
version	0.1.0
kind	paper-skill
quality	positioning-skill-not-executable-science
harness_agnostic	true
layers	{"scientific_invariant":true,"executable_protocol":true,"adapter_binding_examples":false,"research_generation_affordance":true}
paper	{"title":"AI Co-Scientist for Ranking: Discovering Novel Search Ranking Models alongside LLM-based AI Agents with Cloud Computing Access","first_author":"Wu, L.","authors":["Liwei Wu","Cho-Jui Hsieh"],"year":2026,"venue":"arXiv preprint","doi":null,"arxiv_id":"2603.22376","ads_bibcode":null}
domain	{"primary_theme":"other","secondary_themes":["agent-runtime","agent-runtime-full-pipeline-cloud"],"missions":[],"regime":[]}
trigger_keywords	["domain-specific-ai-scientist","agent-runtime-full-pipeline-cloud","agent-runtime","ai-scientist","positioning","agent-runtime-positioning","arxiv:2603.22376"]
data_products	[]
algorithms	[{"name":"Single-LLM routine agents"},{"name":"Multi-LLM consensus agents for analysis + idea generation"},{"name":"Cloud GPU training job scheduling"},{"name":"Expert-in-the-loop pipeline orchestration"}]
validation_target	null
links	{"doi_url":null,"arxiv_url":"https://arxiv.org/abs/2603.22376","ads_url":null,"code_repo":null,"data_repo":null}
claim_boundary	{"scope":"Bounded to commercial-search-ranking research with the named cloud + LLM consensus stack; expert-in-the-loop is required for the headline result.","out_of_scope":["Do not assume the multi-LLM-consensus pattern transfers to all phases without ablation.","Do not generalize 'first study in ranking community' beyond ranking research."]}
failure_modes	["Multi-LLM consensus is expensive; ablations vs. best-of-N or judge-based selection needed.","Cloud-coupling makes the pattern depend on provider availability."]
depends_on	[]
adapter_notes	[]
research_generation_affordances	[{"type":"hypothesis","statement":"HelioSI can borrow the routine-vs-challenging split: routine = data fetch / plot / file IO (single-LLM); challenging = paper-skill cross-reference / hypothesis generation / benchmark interpretation (multi-LLM consensus)."},{"type":"minimal_experiment","statement":"Pilot multi-LLM consensus on one HelioSI challenge — e.g. interpreting a discrepancy between [[paper-sioulas-2023-anisotropic-scaling]] and [[chen-2022-magnetic-field-spectral-evolution-inner-heliosphere]] — and measure agreement vs. expert (HelioSI PI)."}]
provenance	{"generated_by":"HelioSI paper-to-skill factory (Claude Opus 4.7) - wave500_agent_runtime_eval_design_045","generated_at":"2026-05-18T00:00:00Z","source_record":"sioulas-reproduction/results/paper_skill_corpus/wave500_agent_runtime_eval_design_045/manifest.json and sioulas-reproduction/results/agent_runtime_paper_scan_raw.md / agent_runtime_2026_only_synthesis.md (arxiv:2603.22376, doi:n/a)","verified_by":null,"verified_at":null}
tags	["agent-runtime","paper-skill","positioning","ai-scientist","scientific-discovery-evaluation"]

AI Co-Scientist for Ranking: Discovering Novel Search Ranking Models alongside LLM-based AI Agents with Cloud Computing Access - paper-skill (wave500)

Skill type: positioning / evaluation / design-pattern-extractor. This is NOT a heliophysics method. Loaded by the HelioSI runtime (or any agent harness) to (a) audit manuscript/benchmark/positioning claims, (b) reuse design patterns from non-heliophysics AI-scientist work, and (c) seed research-generation affordances for heliophysics-specific runtime upgrades. See §9 for the Layer-4 affordances.

Source type: agent-runtime-positioning. Executable status: design-pattern-extractor.

1. Trigger (Layer 1)

Invoke this skill when an agent (HelioSI manuscript writer, benchmark designer, runtime architect, or any other harness consumer) is:

drafting/revising HelioSI manuscript sections that compare against domain-specific-ai-scientist systems
designing a HelioSI benchmark or evaluation that should reflect the agent-runtime-full-pipeline-cloud pattern
deciding whether a HelioSI subsystem should borrow this paper's design choices or explicitly reject them

Do NOT use this skill as a heliophysics data-pipeline component - it has no data_products[] and no executable scientific workflow over mission data.

2. Paper claim -> verifiable task (Layer 1)

Claim (narrow form). An AI Co-Scientist framework automates the full search-ranking research pipeline (idea -> code -> GPU training -> analysis) with single-LLM agents for routine tasks and multi-LLM consensus agents (GPT 5.2, Gemini Pro 3, Claude Opus 4.5) for challenging phases such as results analysis and idea generation, with expert in the loop and cloud computing access.

Verifiable task. A reproduction of this skill's positioning value succeeds when an agent restates the claim above with the claim_boundary.scope bounds intact, refuses overclaims listed in §7, and emits the §9 Layer-4 affordances in a form a graph walker can consume.

3. Methods / equations -> executable protocol (Layer 2, abstract)

Each item below is the paper's method as an abstract capability the runtime would need to reproduce or borrow the design pattern. No specific runtime (LingTai, Claude Code, an MCP server) is named here - those belong in §8 / adapter_notes[], which are intentionally empty for this wave500 batch.

Method 1: Single-LLM routine agents

Paper role: explicit named method/component of the paper's contribution.
Abstract procedure: the runtime that wants to borrow this pattern must be able to (a) instantiate the component as a callable, (b) feed it the manuscript/benchmark/sibling-skill inputs from §4, and (c) emit an artifact a Layer-4 affordance can reference.

Method 2: Multi-LLM consensus agents for analysis + idea generation

Paper role: explicit named method/component of the paper's contribution.
Abstract procedure: the runtime that wants to borrow this pattern must be able to (a) instantiate the component as a callable, (b) feed it the manuscript/benchmark/sibling-skill inputs from §4, and (c) emit an artifact a Layer-4 affordance can reference.

Method 3: Cloud GPU training job scheduling

Paper role: explicit named method/component of the paper's contribution.
Abstract procedure: the runtime that wants to borrow this pattern must be able to (a) instantiate the component as a callable, (b) feed it the manuscript/benchmark/sibling-skill inputs from §4, and (c) emit an artifact a Layer-4 affordance can reference.

Method 4: Expert-in-the-loop pipeline orchestration

Paper role: explicit named method/component of the paper's contribution.
Abstract procedure: the runtime that wants to borrow this pattern must be able to (a) instantiate the component as a callable, (b) feed it the manuscript/benchmark/sibling-skill inputs from §4, and (c) emit an artifact a Layer-4 affordance can reference.

4. Data / instruments -> abstract tool contracts (Layer 2, abstract)

This paper is a positioning / evaluation / design-pattern source, not a heliophysics-data workflow. The "data" the skill consumes is the HelioSI manuscript draft, benchmark spec, runtime architecture description, or sibling paper-skills.

Input artifact	Role	Capability requirement
HelioSI manuscript draft (`heliosi_manuscript_draft.md`)	Target of positioning checklist	Filesystem read
HelioSI benchmark spec	Target of benchmark-design template	Filesystem read
Sibling paper-skills (this corpus)	Source of cross-references	Skill-catalog read
External paper (DOI/arXiv)	Source for full-text verification (currently TODO)	Web fetch (optional)

5. Validation target -> benchmark artifact (Layer 2)

Not benchmarked yet - this is a positioning-skill-not-executable-science entry. Promotion requires either (a) a manuscript-coverage audit artifact, (b) a benchmark-construction prototype, or (c) a Layer-3 adapter that wires the design pattern into HelioSI and runs it end-to-end on a heliophysics test case.

6. Failure modes -> skill memory (Layer 1)

Pitfalls a future agent applying this skill must remember:

Multi-LLM consensus is expensive; ablations vs. best-of-N or judge-based selection needed.
Cloud-coupling makes the pattern depend on provider availability.

7. Claim boundary (Layer 1)

In scope. Bounded to commercial-search-ranking research with the named cloud + LLM consensus stack; expert-in-the-loop is required for the headline result.

Out of scope - do NOT generalize beyond:

Do not assume the multi-LLM-consensus pattern transfers to all phases without ablation.
Do not generalize 'first study in ranking community' beyond ranking research.

If a downstream task asks for a generalization listed above, refuse it and return a reference to a sibling paper-skill that covers it (or report none).

8. Links and adapter binding examples (Layer 3, optional)

DOI: n/a
arXiv: arXiv:2603.22376
ADS: n/a
Code: n/a
Data: n/a

Adapter notes intentionally empty for this batch - the skill remains executable from §3-§5 alone on any agent harness that supports filesystem reads + skill-catalog walks.

9. Skill graph + research-generation affordances (Layer 4 and graph edges)

Skill graph (depends_on edges).

No paper-skill dependencies (self-contained positioning/design-pattern source).

Research-generation affordances - how this paper helps HelioSI (or any heliophysics agent) generate or evaluate new scientific ideas:

Hypothesis - HelioSI can borrow the routine-vs-challenging split: routine = data fetch / plot / file IO (single-LLM); challenging = paper-skill cross-reference / hypothesis generation / benchmark interpretation (multi-LLM consensus). Related: n/a.
Minimal_experiment - Pilot multi-LLM consensus on one HelioSI challenge — e.g. interpreting a discrepancy between [[paper-sioulas-2023-anisotropic-scaling]] and [[chen-2022-magnetic-field-spectral-evolution-inner-heliosphere]] — and measure agreement vs. expert (HelioSI PI). Related: n/a.

Notes

This SKILL.md is a wave500_agent_runtime_eval_design_045 batch entry. It is harness-agnostic by construction; do not bind it to a specific runtime without converting Layer-3 adapter_notes[] to populated.
The provenance.source_record field points to the inventory anchor used to write this skill; full-text verification of all identifiers is pending and any TODO_verify_with_full_text token must be resolved before promoting past stub tier.