| name | chaining-evidence-rl |
| title | Chaining the Evidence: Robust Reinforcement Learning for Deep Search Agents with Citation-Aware Rubric |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.06021 |
| keywords | ["search-agents","reinforcement-learning","factual-grounding","hallucination-prevention","reward-design"] |
| description | Train search agents using citation-aware rubric rewards that decompose complex questions into verifiable single-hop facts. Agents learn to chain evidence through explicit source citations, preventing hallucinations and shortcut exploitation. Citation-aware Group Relative Policy Optimization (C-GRPO) combines rubric and outcome rewards, enabling agents to solve multi-hop reasoning tasks with high factual grounding and transparency. |
Problem
Current search-based agents face three critical weaknesses:
- Hallucination-Prone: Agents fabricate supporting evidence or skip justification when rewarded only for correct final answers
- Shortcut Exploitation: Agents learn to guess answers without actual reasoning, failing on variations
- Non-Transparent Reasoning: Final answers lack verifiable chains of evidence, making it impossible to audit agent logic
- Weak Generalization: Agents trained on outcome rewards alone don't transfer to open-ended research tasks
Search agents need fine-grained supervision that rewards the reasoning process, not just final outcomes.
Solution
Chaining the Evidence introduces Citation-Aware Rubric Rewards (CaRR):
- Rubric Decomposition: Break complex questions into "verifiable single-hop rubrics"
- Example: "Find the birth year of Obama's 2012 campaign manager" → [Identify campaign manager, Find birth year]
- Citation Requirements: Each rubric element must be grounded in a specific source document
- Evidence Chains: Agents must construct explicit chains linking query → rubric elements → citations → answer
- C-GRPO Training: Combine rubric rewards (process-level) with outcome rewards (final result) during RL training
When to Use
- Complex Reasoning Searches: Multi-hop questions requiring agents to chain facts
- Fact-Checking Agents: Tasks demanding transparent evidence for every claim
- Research Assistance: Agents must cite sources for generated conclusions
- High-Stakes Applications: Medical, legal, financial domains requiring audit trails
- Open-Ended Reasoning: Tasks beyond closed-domain benchmarks with known answers
When NOT to Use
- For simple fact lookup (single-hop retrieval is more efficient)
- When citation data is unavailable or unreliable
- In real-time systems where rubric decomposition adds latency
- For agents that don't need explanation (pure performance optimization)
Core Concepts
The framework operates on the principle that reasoning transparency enables robustness:
- Rubric Design: Break complex queries into atomic facts that can be verified independently
