gpt-pro-research-algorithm-reviewer

name	gpt-pro-research-algorithm-reviewer
description	Deep GPT Pro algorithm/research/pipeline review. Use for RL, reward modeling, OPD, agentic workflows, search/QA, training/eval pipelines, or algorithm proposals where Codex needs stronger hypothesis, experiment, ablation, data leakage, novelty, and go/no-go analysis.

GPT Pro Research Algorithm Reviewer

Target deep algorithm review, experiment design, and research-level critique rather than ordinary code review.

Role split

• Codex: reads repo, builds context bundle, operates GPT Pro, verifies suggestions against code, implements safe changes.
• GPT Pro: acts as an adversarial algorithm reviewer, experiment designer, and paper/research critic.

Do not let GPT Pro directly drive code edits. Treat GPT Pro output as an external review that must be checked against the repo.

Workflow

1. Clarify the algorithm question:
   • What is the task?
   • What is the new idea or current pipeline?
   • What decision does the user need: go/no-go, implementation plan, experiment plan, paper angle, bug diagnosis?
2. Use $bundle-algorithm-context to create a minimal bundle and immutable Codex snapshot.
3. Read and submit references/deep_algorithm_review_prompt.md through $gpt-pro-question-window.
4. Capture the raw answer, then validate its structure against references/review_report_schema.md.
5. Re-open relevant local files and verify GPT Pro's claims.
6. Record the verification as a separate Codex verdict on the same bridge thread.
7. Return a Codex-side synthesis:
   • Valid suggestions.
   • Hallucinated or inapplicable suggestions.
   • Minimal experiments.
   • Implementation checkpoints.
   • Go/no-go judgment.

Completion criterion: every actionable claim is locally classified, missing report sections are identified, and the raw answer and Codex verdict remain separate artifacts.

Required GPT Pro review dimensions

GPT Pro must cover:

1. Problem framing.
2. Core hypothesis.
3. Method decomposition.
4. Strongest arguments for and against.
5. Hidden assumptions.
6. Baseline strength and missing baselines.
7. Data, label, split, and leakage risks.
8. Reward/loss/optimization risks.
9. Agentic/tool/multi-turn pipeline risks, if relevant.
10. Evaluation design and offline-online gap.
11. Concrete ablation matrix.
12. Implementation checkpoints for Codex.
13. Paper angle and novelty, if relevant.
14. Reviewer objections.
15. Minimal experiment plan.
16. Go/no-go judgment.

Special checks for RL / OPD / URM / Agentic QA

If the task involves RL, reward modeling, OPD, user feedback, search/QA, or agents, force these checks:

Reward modeling

• Is the reward explicit, implicit, synthetic, judge-based, or hybrid?
• Are weak feedback signals over-interpreted?
• Does reward aggregation hide conflicts across dimensions?
• Is there length, citation, style, verbosity, or tool-use reward hacking risk?
• Is reward calibrated across query types and user segments?

On-policy / off-policy training

• Are rollouts generated by the current student?
• Is training distribution close to inference-time distribution?
• Is filtering changing distribution too aggressively?
• Are credit assignment signals dense enough?

OPD

• Is teacher feedback token-level, step-level, trajectory-level, or final-answer level?
• Does teacher supervise student mistakes under student-induced states?
• Is dense supervision used where sparse reward would fail?
• Are multiple teachers conflicting?
• Is KL direction consistent with the desired behavior?

Agentic workflow

• Is failure from planning, tool selection, tool execution, observation reading, evidence selection, or final answer synthesis?
• Is retrieval quality separated from answer quality?
• Are tool traces logged and evaluated?
• Is credit assignment step-level or only final-answer-level?

Codex-side verification after GPT Pro answers

Before implementing:

• Check whether GPT Pro referenced files or behavior not in the bundle.
• Check whether proposed ablations are feasible in the current codebase.
• Check whether proposed metrics actually exist or need to be added.
• Check whether any suggestion conflicts with user constraints.
• Separate immediate changes from future research directions.

Final Codex answer must include:

## GPT Pro Review Saved At
<path>

## Codex Verification
- Valid:
- Partially valid:
- Not applicable / hallucinated:

## Minimal Experiment Plan

## Code / Config Changes

## Validation Checklist

## Go / No-Go