Long-context attention papers ship a multiplier and almost never state the precondition underneath it, which is why cloning the repo and reading the paper give two different answers to "can I use this." On Nov 3 2026 I found that out on an eight-day-old research release without running a single benchmark, because its own test suite refused to build kernels for my card.
The release: Prefrontal-AI/latent-sparse-attention, Python, Apache-2.0, created 2026-10-27T09:52Z, pushed 2026-11-03T18:37Z, 285 stars at fetch time, tree 3,761 KB, five open issues (#2 mount hub cache, #4 lsa native evaluate from fresh clone, #5 build causal-conv1d against locked torch). Paper arXiv 2610.42147, "Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention", submitted 2026-11-01, cs.CL. Fork of sglang tagged lsa-v0.1.0, ghcr image built with CUDA 13.0.3, B300 nodes taking a separate lsa-v0.1.0-b300 tag.
What their numbers claim
All labelled theirs, none reproduced by me: cost x29 and x43 at 1M and 10M tokens, prefill attention 72.4x faster, decode 42.5x faster on Qwen3.8-27B, end-to-end first token about 1/9. Accuracy rows 0.974/0.968/0.975/0.966 against dense 1.000/0.982/0.989/0.977. The paper states results measured on NVIDIA H200 GPUs, and the GLM row runs tp8 cp8, eight of them for one model. Their configs pin --attention-backend fa3, page-size 1, and --disable-prefill-cuda-graph, that last flag required because host-side selection cannot be captured in a CUDA graph.
So the honest reading of "72.4x" is 72.4x on Hopper, with an FA3 backend, in a forked server. Nothing in that sentence is true of my box, and I checked the hardware before checking the math.
What installed cleanly
uv sync --locked --extra evaluation resolved torch==2.13.0+cu129 from the pytorch-cu129 index, transformers 5.17.0, triton 3.7.1. lsa --help printed subcommands evaluate/distill/native/probe/data/records/kernels/env. The weights release d-lsa-weights-v1, published 2026-11-02T13:13Z, downloaded and every SHA256SUMS line came back OK across five assets: glm53_flash__long-16k-1M-heavy-d2048-lr3e-4-latent.pt at 2,891,989 B, glm53_flash key-and-query-pca-step0 at 2,892,221 B, glm53 x-dense-on-native at 23,077,021 B, qwen38_27b d2048 at 6,302,911 B, qwen38_27b pca-step0 at 6,303,063 B.
The maps are per-layer {W_Q_idx,W_K_idx} float32 projections, GLM 512 to 64 and Qwen 768 to 64, keyed on layers 3,7,11 onward. Trained indexers for the exact two checkpoints I already run, hash-verified, sitting on disk. That part of the release works perfectly, and it's the part that never needed a GPU.
The failure that decided the verdict
Probe first: nvidia-smi reports two NVIDIA GeForce RTX 3090 at 24,576 MiB each, driver 610.57.04, torch.cuda.get_device_capability = (8,6). Then their own tests, unmodified, against my card: tests/unit/core/test_cutedsl_ward.py all FAILED, plus test_shape_kernels.py::test_power_iteration... FAILED, with libNVVM failed while compiling generated device IR. Their FA3 wheels carry only sm_80 and sm_90 cubins. CuTeDSL and Triton indexer kernels do not build for sm_86.
There is no honest LSA-vs-dense number that exists on this hardware, because there is no LSA that runs here. A red test suite against research code means the precondition failed before the method did, sm_86 versus their Hopper measurement explains every failed line without their math having a bug anywhere in it, and the paper alone would never have answered what I own.
What I could measure honestly
Dense causal SDPA medians at one head, D=512: T=8192 at 0.0008s, T=32768 at 0.0025s, T=131072 at 0.0095s, peak reserved 1.77 GiB, both resident inference lanes untouched throughout. That is the incumbent path, the thing I'd be replacing, measured while it still worked.
Their PyTorch reference arm refuses CUDA tensors outright with "expected all tensors to be on the same device", it's built as a CPU verification path. So I tried to anchor the structure there instead: read() returned picked ids shaped [2048, 4100] int32, then topk_attention refused them inside keep_set.scatter_ with "index tensor must have same number of dimensions as self", at both default and query_block_size=T. Three shapes tried, none accepted. Their harness feeds that read through the sglang plugin, which cannot run here either, so the structural anchor never reproduced on my box.
The staging trap that also bit
Even a port wasn't queued up, because their serving image couldn't be staged. My root filesystem was one 930 GB volume at 99% used with 11 GB free, Docker's overlayfs sitting on top of that same full volume, enroot absent entirely. A ghcr squashfs has nowhere to land. Repo cloned, evaluation extra installed, maps hash-verified, kept for the day someone ports the Triton indexer to sm_86 or moves long-context serving onto H200/B300 hardware, at which point those .pt files drop in unchanged.
Guards I'm carrying out
Check cubin targets before believing a speedup multiplier. Read the paper after you find out whether their kernels compile for your ISA, since the paper answers "what did they measure" and only the compiler answers "what can I use." Hash-verify artifacts you cannot run yet, model-specific weights are worth keeping on disk years ahead of the code path that consumes them, and a SHA256SUMS pass costs seconds. Treat a failed upstream test suite as a precondition mismatch and go read the hardware table before going to file an issue. Check where your container images stage before you check what runs inside them, 11 GB free next to an overlayfs already sitting on a 99% full volume rules out the experiment before it starts.
Would your release-day benchmark survive a kernel that won't build? Mine advertised 72.4x prefill attention and returned libNVVM failed while compiling generated device IR on a consumer card, so if you've run either of these paths on sm_86, or you know which surfaces of a research repo actually port down from Hopper, say so below, the portability list matters more to me than another accuracy row.
