learning-based-robust-control-free-energy

# Learning-Based Robust Control via Free Energy Principle ## Overview Paper: arXiv:2603.06831 (2026-03-05) Authors: Jesawada, Hozefa; Russo, Giovanni; Swikir, Abdalla; Abu-Dakka, Fares This work proposes a **distributionally robust free energy principle** for reliable robotic control. The model jointly learns environment dynamics and rewards while ensuring robustness to epistemic uncertainties. It modifies the maximum diffusion learning framework and validates on continuous-control benchmarks, including real-world Franka Research 3 arm manipulation. ## Key Contributions 1. **Distributionally Robust Free Energy Principle** — Extension of the free energy principle that explicitly accounts for distributional uncertainty 2. **Joint Dynamics + Reward Learning** — Simultaneously learns environment dynamics and reward functions 3. **Epistemic Uncertainty Robustness** — Policies are robust to uncertainties in both environment and reward models 4. **Sim-to-Real Transfer** — Narrows the sim-to-real gap with zero-shot deployment 5. **Real-World Validation** — Demonstrated on Franka Research 3 arm for tabletop manipulation ## Free Energy Principle in Control ### Background The free energy principle (FEP) from computational neuroscience posits that biological systems minimize a variational free energy bound on surprise. In control: - **Perception**: Inferring hidden states from observations - **Action**: Selecting actions that minimize expected free energy - **Exploration vs. Exploitation**: Naturally balances information gathering and goal achievement ### Distributionally Robust Extension The proposed modification introduces distributional robustness: - **Epistemic Uncertainty**: Uncertainty about the true model parameters - **Distributional Robustness**: Policies perform well across a family of plausible distributions - **Ambiguity Set**: Defines the set of distributions the policy must be robust against ## Maximum Diffusion Learning Modification ### Key Modification The framework modifies maximum diffusion learning by: 1. **Explicit Robustness Characterization**: Proves policy robustness to epistemic uncertainties in dynamics and reward 2. **Joint Learning**: Learns dynamics model and reward function together 3. **Distributional Robustness**: Incorporates worst-case analysis over ambiguity sets ### Policy Computation ``` Observations → Joint Model Learning (dynamics + reward) → Free Energy Minimization → Robust Policy → Action ``` ## Theoretical Guarantees - **Robustness to Epistemic Uncertainty**: Explicit characterization of policy robustness bounds - **Distributional Robustness**: Performance guarantees across distributional shifts - **Convergence**: Theoretical convergence properties under the modified framework ## Experimental Validation ### Simulation Benchmarks - Continuous control tasks (MuJoCo, etc.) - Comparison with baselines (PPO, SAC, model-based methods) ### Real-World Experiments - **Platform**: Franka Research 3 robotic arm - **Task**: Tabletop manipulation - **Results**: Repeatable manipulation without task-specific fine-tuning - **Sim-to-Real**: Zero-shot deployment with narrowed sim-to-real gap ## Applications - **Robotic Manipulation**: Reliable control with learned models - **Sim-to-Real Transfer**: Bridging simulation-to-reality gap - **Adaptive Control**: Systems that learn and maintain robustness - **Autonomous Systems**: Reliable deployment in uncertain environments ## When to Use This Skill - Developing robust learning-based controllers for robotics - Addressing sim-to-real transfer challenges - Working with systems where model uncertainty is significant - Applying free energy principle to control problems ## Related Work - **Free Energy Principle**: Karl Friston's active inference framework - **Distributionally Robust Optimization**: Worst-case optimization over ambiguity sets - **Model-Based RL**: Learning dynamics models for planning - **Maximum Diffusion Learning**: Diffusion-based policy optimization ## References - **Paper**: Jesawada, H., Russo, G., Swikir, A., Abu-Dakka, F. "Learning-Based Robust Control: Unifying Exploration and Distributional Robustness for Reliable Robotics via Free Energy," arXiv:2603.06831, Mar. 2026 - **Related**: Free energy principle, active inference, distributionally robust optimization, robotic control