AI & ML
Detecting Silent Faults at the Environment-Trainer Boundary in Reinforcement Learning
arXiv cs.SE0 views
arXiv:2611.08144v1 Announce Type: cross Abstract: Robot reinforcement learning increasingly runs environments outside the trainer: in another process, on another machine, in another language. Many faults at this boundary raise no error. We make the boundary a wire protocol and ask what notices a fault in it. On Pendulum, HalfCheetah, Hopper and a 7-joint arm (Pusher), we injected up to 23 faults into what crosses the wire and 2 into the trainer's log, from one value to every value. Unless every value was wrong, a learning-curve threshold caught at most one run in ten, and a comparison with ten fault-free seeds at most 21%. A clean evaluation of all 1,060 final policies agreed with the curves: of the 728 boundary-fault runs both curve checks passed, 723 scored no more than three standard deviations below fault-free seeds, though nearly all ended on other weights and, for reward faults, logged other returns. Checks of the run itself, not of its outcome, saw every fault. A step ledger kept on both sides and compared step by step caught and located every fault that crossed, on Pendulum, Hopper and Pusher on one machine and on Pendulum across two, with no fault-free run and, on Pendulum, a median 3% more training time. An episode record kept at the environment caught the faults in the trainer's log. We list such faults found in practice, in Gymnasium 1.3 and our own code and, reported by others, in five multi-agent libraries, and recommend a step ledger and episode counts kept at the environment.
Source:https://arxiv.org/abs/2611.08144PreviousSymmetry-Aware Surrogate-Assisted Testing of UAVsNextQDiff: Differential Testing of Quantum Software Stacks
Related posts
How to Plan a Healthtech Product Video Without Sounding Like a Brochure
0dev.to Design
Building a Stress Relief Toy Brand From Wholesale Sourcing to Custom Molds
0dev.to Design
Người trẻ lễ Phật trên điện thoại: trào lưu đi chùa online lan nhanh
0dev.to Discuss

My AI agent says "all fixed." 3 times out of 4, it never measured the before.
0dev.to Discuss

Automating Microstock Content Creation: From Prompt Engineering to Metadata Generation
0dev.to Productivity

How to remove the silent pauses from a voice recording all at once
0dev.to Productivity