AI & ML
Expert-Guided Online Reinforcement Learning for Stable and Sample-Efficient Control of Dynamical Systems
arXiv eess.SY0 views
arXiv:2511.11131v2 Announce Type: replace Abstract: Online reinforcement learning (RL) often requires extensive interactions with the environment, which makes the training costly. Additionally, exploration can destabilise the dynamical system, which may be unsafe. On the other hand, offline learning from expert demonstrations avoids unsafe exploration but is limited by the quality and coverage of the offline dataset. In this paper, we propose a model-free RL approach that combines expert demonstrations with limited online interactions. Specifically, we optimise a policy objective consisting of an online performance term and a behavioural cloning regulariser computed from offline expert demonstrations. The regulariser encourages the learned policy to stay close to expert behaviour, reducing the need to explore unfamiliar actions. Expert demonstrations guide policy learning and reduce the need for many online interactions and exploration. For stochastic linear time-invariant (LTI) systems, we establish conditions for linear convergence of exact policy gradient updates and for all intermediate policies to remain stabilising. Although our theoretical analysis focuses on linear systems, we also evaluate the proposed approach on nonlinear systems to demonstrate its effectiveness beyond the linear setting. \rev{Numerical results show that the proposed method reaches comparable performance in fewer gradient steps and with less number of samples than a purely online RL baseline.
Source:https://arxiv.org/abs/2511.11131PreviousIntegrating Planning and Predictive Control Using the Path Feasibility GovernorNextStrong nonlinear detectability and moving horizon estimation for nonlinear systems with unknown inputs
Related posts

Amazon’s Zoox robotaxis are now available via Uber
0The Verge
Zoox robotaxis are now on the Uber app in Las Vegas
0TechCrunch
Freshworks alums raise $53M to take on Salesforce, HubSpot with AI-native CRM
0TechCrunch
Making infrastructure inspections safer and more reliable
0Phys.org Tech

Replacing workers with AI could prove incredibly costly if the technology doesn't work out
0TechRadar

Campaign to reinstate Huang as Nvidia CEO now has 175,000 signatures
0TechRadar