survival-reinforcement-learning

name	survival-reinforcement-learning
description	Survival Reinforcement Learning (SRL) - online classification-based self-supervised RL that maximizes agent dwell time at target goals, extending survival value learning framework. Bypasses contrastive RL constraints and mitigates "bang-bang" control issues.

Survival Reinforcement Learning (SRL)

Paper: arXiv:2605.31273 | Submitted: 29 May 2026

Authors: Franki Nguimatsia-Tiofack, Fabian Schramm, Théotime Le Hellard, Justin Carpentier

Core Concept

SRL is an online classification-based alternative to Contrastive Reinforcement Learning (CRL) that extends the survival value learning framework. Instead of contrastive losses, SRL maximizes the agent's dwell time at target goals, providing a scalable approach to self-supervised RL.

Key Innovation

• Classification-based formulation - bypasses structural constraints of contrastive losses
• Survival value learning - maximizes time spent at goals rather than distance minimization
• Mitigates "bang-bang" control - addresses undesirable behaviors in complex dynamical systems
• Depth-scalable - supports networks over 64 layers (like CRL)
• Long-horizon capable - outperforms CRL on stable, long-horizon locomotion tasks

Implementation Framework

1. Survival Value Objective

V_survival(s, g) = max E[∫_0^T 1_{at_goal} dt]

Maximize expected dwell time at goal state g starting from state s.

2. Classification Formulation

Instead of contrastive loss comparing goal vs non-goal states:

L_SRL = -log P(τ reaches goal | τ from policy)

Online classification: "Does trajectory reach/sustain goal?"

3. Key Differences from CRL

Aspect	CRL	SRL
Loss Type	Contrastive (pull goal, push non-goal)	Classification (goal reach prediction)
Scaling	Uniformity-tolerance dilemma	No contrastive constraints
Control	Can induce "bang-bang" behavior	Smooth, stable control
Long-horizon	Struggles	2x-8x improvement

4. Training Procedure

1. Goal conditioning: Sample goal states g
2. Trajectory generation: Roll out policy π(a|s,g)
3. Survival prediction: Classify trajectory success (reached/sustained goal)
4. Update: Maximize survival probability via gradient

Performance Results

• Manipulation tasks: Matches state-of-the-art CRL
• Locomotion tasks: 2x to 8x improvement over CRL
• Long-horizon planning: Stable, sustained goal achievement
• Depth scaling: Works with 64+ layer networks

Applications

Primary Use Cases

1. Self-supervised RL - no reward engineering required
2. Goal-conditioned planning - long-horizon tasks
3. Robotic locomotion - stable, continuous control
4. Robotic manipulation - goal-reaching tasks

When to Use SRL

• Long-horizon goal-conditioned tasks
• When contrastive RL struggles with uniformity
• Stable locomotion with sustained goals
• Self-supervised learning without explicit rewards

Technical Details

Mitigation of "Bang-Bang" Control

Traditional survival frameworks induce "bang-bang" solutions (extreme control switching). SRL's classification formulation smooths this:

• Soft goal prediction instead of hard distance minimization
• Continuous survival probability estimation
• Gradual policy adaptation

Sufficient Statistics

SRL computes survival value as sufficient statistic for:

• Optimal policy learning
• Goal-conditioned decision making
• Long-horizon planning

Activation Keywords

• survival reinforcement learning
• SRL self-supervised
• goal dwell time
• classification-based RL
• contrastive RL alternative
• long-horizon goal conditioning
• stable locomotion RL

Related Skills

• [[model-based-diffusion-policy-optimization]] - Diffusion-based policy optimization
• [[efficient-tdmpc]] - Model-based RL
• [[sbsrl-sampling-based-safe-rl]] - Safe RL methods
• [[lilac-safe-continual-rl]] - Continual safe RL

References

• Paper: arXiv:2605.31273 - "Survival Reinforcement Learning: Toward Scalable Self-Supervised RL"
• Contrastive RL: prior depth-scaling work
• Survival value theory: foundational RL theory

---

Category: reinforcement-learning, self-supervised-RL, goal-conditioned