| name | smart-search-query-refinement |
| title | SmartSearch: Process Reward-Guided Query Refinement for Search Agents |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.04888 |
| keywords | ["search-agents","query-optimization","reinforcement-learning","process-rewards","knowledge-intensive-tasks"] |
| description | Improve LLM search agents by optimizing query quality at each step using process-level rewards. Framework teaches agents to iteratively refine search queries through imitation, alignment, and generalization stages. Agents learn to identify low-quality queries and regenerate improved ones, significantly improving search efficiency and answer quality on knowledge-intensive reasoning tasks. |
Problem
Search agents frequently generate inaccurate or imprecise search queries, leading to:
- Poor Retrieval Results: Queries that fail to capture task intent retrieve irrelevant documents
- Inefficient Search: Agents waste tokens on multiple search iterations due to query misalignment
- Cascading Errors: Low-quality queries compound through multi-hop reasoning tasks, degrading final answer quality
Current search agents focus on reasoning paradigms (chain-of-thought, self-reflection) rather than optimizing the quality of individual search queries—the actual tool at the core of retrieval.
Solution
SmartSearch introduces Process Reward-Guided Query Refinement:
- Dual-Level Credit Assessment: Evaluate search query quality with fine-grained supervision, distinguishing between queries that retrieve relevant documents vs. those that miss key information
- Iterative Query Refinement: Identify low-quality queries during generation and regenerate improved versions before executing search
- Curriculum Learning Pipeline: Train agents through three stages:
- Imitation: Learn from high-quality query examples
- Alignment: Optimize query generation using process rewards
- Generalization: Generate improved queries independently without explicit guidance
When to Use
- Building search agents for knowledge-intensive QA (research, fact-checking, multi-hop reasoning)
- Improving efficiency of retrieval-augmented generation (RAG) systems with token constraints
- Training agents that must formulate multiple search queries in sequence
- Scenarios where search precision directly impacts downstream reasoning quality
- Multi-turn conversation tasks requiring dynamic information retrieval
When NOT to Use
- For single-query lookup tasks (use static retrieval ranking instead)
- When queries can be directly provided by users (no generation needed)
- In low-resource settings where process reward training is computationally prohibitive
- For retrieval tasks where keyword matching is sufficient
Core Concepts
The framework operates on the principle that query quality is learnable:
- Process Reward Signal: Instead of only rewarding final answer correctness, reward intermediate steps where good queries lead to useful retrieval results
