Technique
GPU Acceleration.
Moving work onto the GPU, where the algorithm and the silicon stop being separable concerns and staying on the fast kernel decides the result.
5 papers · Technique
Lynx: the KV cache does not have to arrive whole
In disaggregated LLM inference the bottleneck is moving the KV cache, not computing on it: a single 128K request can be 23.5 GB, which is two seconds on a 100 Gbps link before decoding can start. A team at University College London and Huawei split the cache into a four-bit anchor stream carrying order of magnitude and a four-bit residual stream carrying precision, start decoding speculatively on the anchor, and verify when the residual lands. A reading of the quantizer that makes the bit-split work, the measured acceptance rate, and why the accuracy gain and the latency gain come from two different halves of the paper.
Read the paper →- October 2026EngineeringAGA-49
ScaleSweep, and what AbsMax leaves on the table in NVFP4
NVFP4 gives every 16 elements their own FP8 scale, and almost every quantiser picks that scale the same way: divide the block maximum by 6. Two researchers at Peking University prove the optimum lies within eleven FP8 bit patterns of that guess, search them, and recover 93.9 per cent of BF16 performance with weights, activations, KV cache and query states all at four bits. A reading of the format, the derived bounds, the cost at each configuration, and why the paper's own ablation suggests the bounds buy latency rather than accuracy.
Luis Hasanaj · Read → - October 2026ResearchAGA-46
Log-linear attention, and what a logarithm buys
Linear attention compresses a whole prefix into one fixed-size state. A team at MIT, Carnegie Mellon and Princeton replaces that single state with a logarithmically growing set, cut by a Fenwick tree, giving O(T log T) training compute and O(log T) decode memory. A reading of the mechanism, the chunkwise algorithm that earns the complexity claim, the long-context gains, and the retrieval tasks in the appendix where it loses to the linear model it extends.
Luis Hasanaj · Read → - September 2026ResearchAGA-25
Attention Is All You Need, nine years on
The 2017 paper every frontier model descends from got its own title wrong. Attention was the mechanism; removing the sequential dependency was the payload. A technical audit of which architectural decisions survived, which were quietly replaced, and why the one flaw nobody solved still shows up on your invoice.
Luis Hasanaj · Read → - January 2026EngineeringAGA-01
GPU Query Engine (GQE): a reference architecture for GPU-accelerated SQL
A reference architecture for a GPU-accelerated SQL engine, built on libcudf and NVCOMP, that pushes complex analytics query processing to speed-of-light on modern GPUs.
Luis Hasanaj · Read →