AI & ML
Patch Attention Wisely: Understanding Patch Length in Attention-Based Forecasting
arXiv stat.ML0 views
arXiv:2611.08160v1 Announce Type: new Abstract: Attention-based time series forecasters commonly group consecutive observations into patches. Patch length is known to have a substantial impact on forecasting performance, yet its selection remains largely empirical. We develop a theoretical framework that explains why patch length matters for forecasting accuracy and provides a principled basis for its selection. For linear state-space models, we derive an exact factorization of the forecasting coefficient matrix that separates forecast dynamics from filtering dynamics. We show that dominance of a single forecast component at long horizons does not generally imply a rank-one representation. This explains why longer horizons alone need not reduce representational complexity. We combine the factorization with estimation bounds to develop a patch-selection criterion that balances representation and estimation errors under temporal dependence. Our analysis establishes conditions under which balanced patching is justified and clarifies how predictive structure, head count, training sample size, and long-run dependence enter the selection criterion.
Source:https://arxiv.org/abs/2611.08160PreviousIdentifying Neural Dynamics Using Nonlinear Interventional State Space ModelsNextSeries-wise Cross-validation for Global Time Series Forecasting
Related posts
Aircraft simulations scale new heights with supercomputer
0Phys.org Tech

AI books to blame after a new public library in India faces investigation
0TechRadar

I've been hunting for early Black Friday phone deals: these are the 22 best on the iPhone Duo, 18 Pro, Samsung Galaxy S26, Pixel 11, and more
0TechRadar

How to watch Czechia vs Spain: FREE live streams, TV channels & kick-off time for Nations League 2026/27 fixture
0TechRadar
Massive atlas of brain region linked to a host of diseases is the most detailed yet
0Nature News

Pixel Watch and Fitbit Air will be able to control Android XR glasses with gestures
09to5Google