AI & ML
Expert-Guided Online Reinforcement Learning for Stable and Sample-Efficient Control of Dynamical Systems
arXiv eess.SY0 views
arXiv:2511.11131v2 Announce Type: replace Abstract: Online reinforcement learning (RL) often requires extensive interactions with the environment, which makes the training costly. Additionally, exploration can destabilise the dynamical system, which may be unsafe. On the other hand, offline learning from expert demonstrations avoids unsafe exploration but is limited by the quality and coverage of the offline dataset. In this paper, we propose a model-free RL approach that combines expert demonstrations with limited online interactions. Specifically, we optimise a policy objective consisting of an online performance term and a behavioural cloning regulariser computed from offline expert demonstrations. The regulariser encourages the learned policy to stay close to expert behaviour, reducing the need to explore unfamiliar actions. Expert demonstrations guide policy learning and reduce the need for many online interactions and exploration. For stochastic linear time-invariant (LTI) systems, we establish conditions for linear convergence of exact policy gradient updates and for all intermediate policies to remain stabilising. Although our theoretical analysis focuses on linear systems, we also evaluate the proposed approach on nonlinear systems to demonstrate its effectiveness beyond the linear setting. \rev{Numerical results show that the proposed method reaches comparable performance in fewer gradient steps and with less number of samples than a purely online RL baseline.
Source:https://arxiv.org/abs/2511.11131PreviousIntegrating Planning and Predictive Control Using the Path Feasibility GovernorNextStrong nonlinear detectability and moving horizon estimation for nonlinear systems with unknown inputs
Related posts
Your Manager's Manager Has Booked a Skip Level Meeting With You
0dev.to

How to Make a Character Reference Sheet with AI (Prompts Included)
0dev.to Design
How Virtual Assistants Help Businesses Manage Their Work More Efficiently
0dev.to Productivity
DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing
0arXiv eess.IV
D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms
0arXiv eess.IV
Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation
0arXiv eess.IV