| name | qnrl-quantum-native-rl |
| description | Quantum-Native Reinforcement Learning (QnRL) methodology for distributional RL using quantum state representations |
| category | quantum-ml |
| source | arxiv |
| arxiv_id | 2606.08276 |
| paper_title | QnRL: Quantum-Native Reinforcement Learning |
| paper_authors | Alexander DeRieux, Walid Saad |
| trigger | ["qnrl","quantum reinforcement learning","quantum native rl","distributional quantum rl","quak algorithm","quantum amplitude kickback","quantum state distribution","hilbert space rl","quantum policy optimization"] |
| version | 1.0.0 |
| created | 2026-06-09 |
QnRL: Quantum-Native Reinforcement Learning
Overview
QnRL (Quantum-Native Reinforcement Learning) is a distributional RL framework that learns conditional distributions naturally in Hilbert space via superimposed and entangled quantum states. Unlike existing QRL architectures that indirectly approximate environment behavior by estimating expected outcomes, QnRL directly models the behavior of stochastic learning environments via the natural properties of quantum systems.
Key Results:
- Up to 82.9% higher evaluation scores compared to baselines
- Up to 94.3% fewer parameters on average
- More accurate estimation of expected return for unseen observations
- Better adaptation to varying stochastic conditions
Core Methodology
1. Quantum State Distribution Modeling
Instead of modeling random variables directly, QnRL represents environment random variables as quantum state distributions in Hilbert space:
$$|\psi(s)\rangle = \sum_{i} \alpha_i(s) |i\rangle$$
where $\alpha_i(s)$ are probability amplitudes encoding the distribution of states.
2. Quantum Amplitude Kickback (QuAK) Algorithm
The core innovation enabling QnRL:
Purpose: Compare the $n$-th power of the $m$-th moment of multiple superimposed distributions entirely within Hilbert space.
Steps:
- Prepare superimposed quantum states representing value distributions
- Apply controlled unitary operations to encode moment information
- Use amplitude kickback to compare distributions without measurement collapse
- Distill conditional action policy distribution from quantum generative model moments
Mathematical Foundation:
- Conditional action policy distribution is distilled from moments of quantum generative model
- Entire optimization occurs within Hilbert space
- Proven theoretically to converge to optimal policy
3. Distributional RL in Hilbert Space
Advantages over classical distributional RL:
- Expressive Power: Extra dimensions for expressing environment correlations unknown to classical models
- Parameter Efficiency: Quantum superposition enables compact representation of complex distributions
- Adaptive Potential: Natural quantum dynamics enable better adaptation to stochastic environments
Implementation Patterns
Pattern 1: Quantum Value Distribution Encoding
