| name | prism-process-reward-guided-inference |
| title | PRISM: Pushing the Frontier of Deep Think via Process Reward Model-Guided Inference |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2603.02479 |
| keywords | ["Process Reward Models","Inference Optimization","Error Correction","Step-Level Feedback","Beam Search"] |
| description | Optimize multi-step reasoning by treating candidate solutions as particles in a process-reward energy landscape. Use PRM step-level scores to guide stochastic refinement and population resampling, achieving directional error correction without hallucination amplification. |
PRISM: Process Reward Model-Guided Inference for Error-Corrective Reasoning
Existing deep reasoning systems struggle with a critical bottleneck: iterative refinement of reasoning traces without correctness signals tends to amplify errors and suppress minority correct solutions. Standard sampling-based approaches treat refinement as random perturbation, losing valuable quality information. PRISM solves this by embedding Process Reward Model (PRM) scores directly into particle evolution dynamics.
The core innovation treats each candidate reasoning trace as a "particle" whose quality is defined by PRM scores at each step. Rather than independent refinement attempts, the system uses score-weighted resampling and probabilistic acceptance criteria to guide the population toward higher-quality solutions while maintaining exploration capacity.
Core Concept
PRISM operates on the intuition that Process Reward Models provide step-level quality signals that standard end-to-end rewards miss. By treating these signals as an energy landscape, the method can:
- Concentrate probability mass on higher-scoring reasoning paths through importance weighting
- Accept score-improving moves while occasionally exploring lower-scoring alternatives (MCMC-style)
- Prevent population collapse by resampling when effective sample size drops, ensuring diversity persists
- Achieve net-positive error correction rather than stochastic drift
Architecture Overview
- Input: Initial reasoning prompt, set of sampled reasoning traces {τ₁, τ₂, ...}
- PRM Scoring: Evaluate each trace via step-level Process Reward Model to get quality scores
- Energy-Based Weighting: Convert scores to importance weights via Boltzmann distribution with temperature parameter
- Particle Refinement: Propose stochastic modifications to traces, accept with probability based on score ratios
- Resampling: Monitor effective sample size (ESS); resample when ESS/K < threshold to restore diversity
- Output: Best-scoring refined trace(s) for downstream verification
Implementation Steps
The method decomposes into discrete refinement iterations, each following an energy-landscape paradigm.
Step 1: Initialize candidate set from language model sampling
Standard beam or temperature-sampled decoding produces an initial population of k candidate traces from the LLM. This population represents diverse reasoning paths from the same prompt.
initial_traces = [llm.sample(prompt, temperature=T) _ (k)]
scores = [prm.score_trajectory(trace) trace initial_traces]
