Frontend
Uncertainty-Aware Hybrid Offline Reinforcement Learning with Online Action Selection Enabled by Sparse Gaussian-Process Critics
arXiv eess.SY0 views
arXiv:2611.14374v1 Announce Type: cross Abstract: Start-ups and grade changes remain largely operator-driven, while developing and maintaining dynamic models for model-based control requires substantial engineering effort. Hybrid Offline Learning and Online Optimisation (HOFLON) uses one-step offline reinforcement learning to select actions from historical transition data. In its original implementation, the online objective combines the predicted return from a tree-ensemble critic with an adversarial-autoencoder (AAE) penalty for unfamiliar state-action pairs. We examine the effect of this penalty and replace the critic and AAE with a sparse variational Gaussian-process (sGP) critic. Actions maximise $\mu - \kappa\sigma$, an uncertainty-adjusted estimate of the return from one action followed by the historical operating practice. On the HOFLON polymerisation-reactor benchmark, removing the AAE from HOFLON led to 4 failed start-ups in 15 runs over five critics. At its published weight, the variation in the AAE penalty is four orders of magnitude smaller than the variation in the tree-ensemble critic. Nevertheless, it changes actions when the predicted returns of competing candidates are nearly equal, and paired runs with and without the penalty diverged from the first move. With a Mat\'ern-5/2 sGP critic, $\kappa = 0$ led to failure in 13 of 15 runs, whereas with $\kappa = 3$ none of 15 runs (five critics, three noise realisations) failed. The median and lowest returns were 13,464 and 13,398 (HOFLON: 13,393 and 12,411). A squared-exponential sGP critic with learned inducing inputs needed $\kappa \geq 3$ to remain within the empirical data support in steady state, and then settled with a temperature offset of 2.0 K.
Source:https://arxiv.org/abs/2611.14374PreviousReliable and explainable analysis of AVS/RS by Statistical Model Checking and Process MiningNextThe Phantom of Davis-Wielandt Shell: A Unified Perspective for Graphical Stability Analysis of MIMO LTI Systems
Related posts
Chrome Extension Flagged as Malware: 7 Steps to Clear It
0dev.to
JSONDecodeError: Expecting value at char 0? Read the Error Position Before You Touch the JSON
0dev.to
Класс пожарной опасности здания как модель данных: почему один enum на здание — ошибка
0dev.to
CFAR Detection on Non-Coherently Integrated Angular Maps for mmWave Radio SLAM
0arXiv eess.SP
Reconciling Strict SLOs with Low Cost under Fine-Grained Load Fluctuations
0arXiv cs.DC
Unified Far-field and Near-field Codebook Design for XL-array Systems
0arXiv math.IT