arXiv:2604.19344v2 Announce Type: replace Abstract: Agile locomotion is important for quadruped robots in the real world. Recent approaches have demonstrated impressive capabilities including dynamic climbing and jumping but typically rely on sequential multilayer perceptron (MLP) architectures with densely activated layers. In contrast, sparsely gated mixture-of-experts (MoE) architectures have become popular in the large language model (LLM) domain as a tool to improving scalability and performance by activating only a subset of parameters at inference. In this work, we investigate the application of such architectures to vision-based robotic locomotion. We compare policies based on standard MLPs and MoE architectures under a controlled setting where the number of active parameters at inference time is matched. Experimental results show an 8\% increase in reward during training comparing the MoE to a standard MLP and even show benefits comparing to MLPs with equivalent total parameters. Experimental results on a real Unitree Go2 quadruped robot demonstrate performance in the real world. These results highlight that sparsely gated MoE architectures may pose advantages in tasks where network capacity is important and that borrowing techniques from the fast-moving LLM space and applying them to locomotion is a promising direction.