Backend
IA-MoE: An Intensity-Aware Mixture of Experts for DNN Inference
arXiv cs.DC0 views
arXiv:2611.19669v2 Announce Type: replace Abstract: In edge-cloud collaborative inference, the layers of a deep neural network (DNN) are placed on edge and cloud virtual machines (VMs) for every request, while the edge load and the cross-cloud bandwidth change over time. We propose IA-MoE, an intensity-aware Mixture of Experts (MoE) framework that scores each layer-VM pair by end-to-end delay and energy and places the layers one at a time on the best-scoring VM. Such greedy placement is myopic when trained on per-layer labels: offloading to the cloud slows the first offloaded layer while the gain appears only in later layers, so the policy keeps too many requests at the edge. Training on the remaining request time after each decision removes this myopia and reduces the mean delay by 12-56% on congested workloads. In CloudSimSDN simulations with three DNNs and two public request traces, IA-MoE has the lowest mean delay of all compared methods on congested VGG16 workloads, across cross-cloud bandwidths of 150-600 Mbps, fluctuating bandwidth and saturated edge servers. A random forest trained on the same label is 5-27% slower, and only a heuristic that sends whole requests to the cloud uses less energy, at up to 20 times the delay. When the edge is lightly loaded, offloading does not pay and simple heuristics perform as well. IA-MoE also beats a Double Deep Q-Network (DQN) agent with the same features in all 19 tested settings, with a mean delay lower by a factor of 1.3 to 10.5, and trains in minutes instead of hours.
Source:https://arxiv.org/abs/2611.19669PreviousWhy Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model ServingNextFlux: Optimal Scheduling of Optical Circuit Switches for LLM Training
Related posts
SQL Server migration verification: matching row counts are only the start
0dev.to Database

The Claim Row Stopped the Duplicate. My Agent Sent the Payment Twice Anyway.
0dev.to Database

SeaweedFS 4.54: S3 Storage, Iceberg Tables and Vector Buckets
0dev.to Database
Where Is the Cost of Third-Party API Routers in Agentic Software Development?
0arXiv cs.SE
RyBench: Runtime Verification Benchmarking Framework over Publish/Subscribe Networks
0arXiv cs.SE
When the World Is the Test Suite: Agentic Intelligence for the Physical World
0arXiv cs.SE