Type
Research Note.
A reading of published work by others, or of our own position, with the sources named and the limits of the evidence stated.
34 papers · Type
Every company needs a model testing manager
Frontier models now ship faster than any procurement cycle can absorb, and the published benchmarks cannot tell you which one is right for your work. Somebody has to own that decision. The argument for two new roles, the one we would not create, and the test that separates a job title that lasts from one that gets absorbed within two years.
Read the paper →- October 2026ResearchAGA-54
Latent distillation needs no shared grid
Distilling a vision encoder into an autoencoder bottleneck makes diffusion converge faster, and everyone did it by matching each latent cell to the teacher cell above it. Adrien Ramanana Rahary and colleagues show that correspondence is unnecessary: one pooled descriptor per image matches it, 5.77 gFID against 6.04. The margin is inside seed noise and the authors say so. The result is not a better score, it is that distillation now reaches 1D latents and text teachers, where the position wise method cannot run at all.
Luis Hasanaj · Read → - October 2026ResearchAGA-42
ViT-5, and what vision should not borrow from language
A team at Johns Hopkins and UC Santa Cruz swept the vision transformer component by component and found that SwiGLU, the gated block essentially every language model uses, is actively harmful in vision when paired with LayerScale. Not because either is wrong, but because both filter channels and stacking them suppresses too much. A reading of the full ablation, what else survives the transplant, and why a component validated alone is not validated.
Luis Hasanaj · Read → - October 2026ResearchAGA-52
Verifier errors inside a GRPO group are not independent
GRPO centres the policy gradient on the mean of a group of rollouts, and that is defended as noise-robust because averaging over k samples should wash out any single verifier mistake. The defence requires the k verifier errors to be independent, and an independent researcher went and measured it: on 24,998 eight-completion groups from Qwen2.5-1.5B, within-group error correlation is 0.530, so an eight-rollout group carries the information of 1.70 independent samples. A reading of the mechanism, the category split that inverts the usual verifier audit, and why raising the group size cannot fix it.
Luis Hasanaj · Read → - October 2026EngineeringAGA-50
Lynx: the KV cache does not have to arrive whole
In disaggregated LLM inference the bottleneck is moving the KV cache, not computing on it: a single 128K request can be 23.5 GB, which is two seconds on a 100 Gbps link before decoding can start. A team at University College London and Huawei split the cache into a four-bit anchor stream carrying order of magnitude and a four-bit residual stream carrying precision, start decoding speculatively on the anchor, and verify when the residual lands. A reading of the quantizer that makes the bit-split work, the measured acceptance rate, and why the accuracy gain and the latency gain come from two different halves of the paper.
Luis Hasanaj · Read → - October 2026ResearchAGA-51
A verifier is a specification, and models optimise it
Two papers on reward hacking under verifiable rewards. Helff and colleagues find RLVR-trained models abandoning rule induction and enumerating instance-level labels that pass the verifier without learning the pattern, and catch it by checking one output under both extensional and isomorphic verification. Mahmoud and colleagues train a policy against a single rubric verifier and score it with a cross-family panel of three frontier judges, where exploitation climbs from 39 to 65 per cent and a strong verifier still loses to the base model on 78.4 per cent of prompts. Read together: a verifier is a partial specification, and the policy finds the part you left out.
Luis Hasanaj · Read → - October 2026EngineeringAGA-49
ScaleSweep, and what AbsMax leaves on the table in NVFP4
NVFP4 gives every 16 elements their own FP8 scale, and almost every quantiser picks that scale the same way: divide the block maximum by 6. Two researchers at Peking University prove the optimum lies within eleven FP8 bit patterns of that guess, search them, and recover 93.9 per cent of BF16 performance with weights, activations, KV cache and query states all at four bits. A reading of the format, the derived bounds, the cost at each configuration, and why the paper's own ablation suggests the bounds buy latency rather than accuracy.
Luis Hasanaj · Read → - October 2026ResearchAGA-53
Reward aggregation is an attack surface
Yu Yuan and colleagues found a reward hacking failure that no single reward permits: the weighted sum that combines them. Between training steps 35 and 40 their scalar reward rose from 1.445 to 1.973 while accuracy fell from 0.258 to 0.090, because format and length saturated as correctness collapsed. Paired with a multimodal study where adding a visual-grounding reward made hacking worse at every model scale, the argument is that each reward dimension you add is a new substitution the policy can make.
Luis Hasanaj · Read → - October 2026ResearchAGA-41
Mixture of experts: what actually changed
Expert counts went from 8 to 896 in under three years, and that is the least interesting thing that happened. The share of a model that runs for any given token fell from 27.6 per cent to 3.7. A reading of the three structural changes that made that possible: experts small enough to compose rather than choose between, a shared expert holding what every token needs, and load balancing moving out of the loss function and into a controller beside it.
Luis Hasanaj · Read → - October 2026ResearchAGA-45
Where a hybrid model puts its massive activations
In a dense Transformer the enormous outlier activations sit at a plateau across most of the depth. Interleave linear attention with full attention, as Kimi Linear, Qwen3.5, Nemotron-H and Zamba2 all do, and they reorganise: a sharp spike in the layer immediately before every full attention layer, with 99.4 to 100 per cent alignment, plus plateaus between spikes once attention gets dense enough. Deleting four of those coordinates per attention layer costs up to 88 per cent of retrieval accuracy. A reading of the write-sink-cancel mechanism, the interventions, and why we think this makes hybrids easier to quantise rather than harder.
Luis Hasanaj · Read → - October 2026ResearchAGA-46
Log-linear attention, and what a logarithm buys
Linear attention compresses a whole prefix into one fixed-size state. A team at MIT, Carnegie Mellon and Princeton replaces that single state with a logarithmically growing set, cut by a Fenwick tree, giving O(T log T) training compute and O(log T) decode memory. A reading of the mechanism, the chunkwise algorithm that earns the complexity claim, the long-context gains, and the retrieval tasks in the appendix where it loses to the linear model it extends.
Luis Hasanaj · Read → - October 2026ResearchAGA-47
How much full attention does a hybrid need?
Two papers answer the same question from opposite ends. A 72-model sweep at UC Santa Cruz and ByteDance says one full-attention layer for every three to six linear ones, because language modelling loss is flat across every ratio and only recall sets a floor. A Tsinghua and OpenBMB group fixes the ratio and shows why: long-range retrieval is carried almost entirely by the full-attention layers, and a large sliding window makes those layers learn to retrieve more slowly. A reading of both, the ablations that localise the mechanism, and our argument that the ratio is the less important variable.
Luis Hasanaj · Read → - October 2026ResearchAGA-44
Gated DeltaNet-2, and the gate that was doing two jobs
Linear attention trades the unbounded KV cache for a fixed-size recurrent state, which makes editing compressed memory the whole problem. The delta rule reads a key's current value and subtracts it before writing the new one, but a single scalar has been scaling both halves of that edit. A team at NVIDIA split it into a channel-wise erase gate and a channel-wise write gate, and multi-key retrieval at 2K context goes from 37.0 to 51.4 per cent. A reading of the mechanism, the ablation that says the erase side carries three quarters of the gain, and what the fixed state still costs at 8K.
Luis Hasanaj · Read → - October 2026EngineeringAGA-48
DeepSeek-V4.1-Flash and the 890-byte KV cache
DeepSeek brings the global KV cache to 890 bytes per token, a quarter of DeepSeek-V4-Flash, by serving forty transformer layers from four shared caches and storing them in FP4. A reading of the actual mechanism: the three CSA2 modes, the Causal Encoder-Decoder that halves prefill, and the bounded replay that is really a cache retention policy. We reconstruct the 890 bytes from the published configuration, and argue the paper's quality costs are asserted rather than measured.
Luis Hasanaj · Read → - October 2026ResearchAGA-38
Kimi K3: where the compute went
Moonshot AI's Kimi K3 technical report makes a compute claim rather than a quality one: roughly 2.5 times more validation accuracy per unit of training compute than Kimi K2, measured recipe against recipe with no component ablations. A reading of Kimi Delta Attention, the 896 expert mixture, the nine specialists merged into one shipped model, and the resumable training harness that is the least glamorous and most transferable idea in it.
Luis Hasanaj · Read → - October 2026EngineeringAGA-37
Gemini 4 Argon: the output limit is the story
Google DeepMind's Gemini 4 Argon raises the output token ceiling from 64,000 to 1,000,000, and that single number changes more about how agent systems are built than any benchmark in the announcement. A reading of what the new ceiling costs, why shipping a model without cyber guardrails to a named cohort makes capability an identity problem, and the one optimisation DeepMind deliberately refused to take.
Luis Hasanaj · Read → - October 2026ResearchAGA-39
What the frontier costs now
Every current frontier and near-frontier model priced against the same two workloads on 1 October 2026, from the providers rather than from aggregators. Gemini 4 Argon and Claude Opus 5.5 turn out to have identical price lists to the cent, three labs deepened the cache read discount in the same quarter, and the whole table carries an error term nobody publishes: a price per token only compares if a token means the same amount of text on both sides, and it does not.
Luis Hasanaj · Read → - September 2026ResearchAGA-31
GPT-6.1 Sol, and the model that was cancelled
OpenAI shipped a model at a fifth of Astra's token price that it says lands within 1.9 percent of it. The larger news is the model it did not ship: GPT-6.1 Astra was held back after internal testing found more deception and a tendency to act without asking permission. Those are the two behaviours the Astra system card measured three weeks earlier.
Luis Hasanaj · Read → - September 2026ResearchAGA-29
When the verifier moves inside the model
Four papers from one week in robotics, image generation and view synthesis all pay for an expensive correctness check during training and then delete it before inference. It is a real efficiency result and it removes the thing you would otherwise have logged. Reading DexAgent, InternW0-Delta, UMM-Reflection and GeoVerse together, and what disappears along with the verifier.
Luis Hasanaj · Read → - September 2026ResearchAGA-28
QiYao-M: two kinds of forecasting signal
A forecasting model from ECNU and Huawei argues that the extra signals you feed a forecaster are not one category. Your own history and the world outside play different roles and deserve different mechanisms. The headline gain over other foundation models is about one percent. The part worth your attention is that a new external signal can be added without retraining anything.
Luis Hasanaj · Read → - September 2026ResearchAGA-27
CLM-8B, Jev, and language model shape
In one week, four independent groups published work that stops treating the language model as a fixed-shape black box: a model that ranks instead of generating, one whose output is constrained to a single number, one that varies depth per token, and one that changes the generation order. Reading them together, and the engineering bill nobody is itemising.
Luis Hasanaj · Read → - September 2026ResearchAGA-25
Attention Is All You Need, nine years on
The 2017 paper every frontier model descends from got its own title wrong. Attention was the mechanism; removing the sequential dependency was the payload. A technical audit of which architectural decisions survived, which were quietly replaced, and why the one flaw nobody solved still shows up on your invoice.
Luis Hasanaj · Read → - September 2026ResearchAGA-23
Why the brain keeps two memory systems
Mammals evolved a fast episodic store and a slow semantic one because collapsing them destroys knowledge. A 2026 result shows AI agents rediscovering that the hard way: consolidating memories from ground truth solutions made a model fail 54 percent of problems it had already solved. The theory that predicts it, and what it means for building agent memory.
Luis Hasanaj · Read → - September 2026ResearchAGA-20
GPT-6 Astra: what the footnotes say
OpenAI calls GPT-6 Astra the world's most intelligent model. Independent measurement puts it level with its own predecessor on the intelligence index and behind Claude Fable 5.1, at 75 percent more per task. Both readings are true, they measure different things, and the difference is the most interesting thing about this launch.
Luis Hasanaj · Read → - September 2026ResearchAGA-22
AGI talk measures the wrong axis
Aggregate intelligence has moved about five points in a year. Token efficiency, scope adherence, computer use and cost per completed task have moved by multiples. The industry keeps announcing progress on the axis that stopped moving and burying the one that did, and the confusion has a technical cause worth setting out.
Luis Hasanaj · Read → - September 2026ResearchAGA-17
Small language models: the agent economics
Most calls an agent makes are narrow, repetitive and tightly specified, which is the profile a small model handles well and a frontier model handles expensively. The case for heterogeneous agent systems, what the evidence actually supports, and how to route calls by task rather than by habit.
Luis Hasanaj · Read → - September 2026ResearchAGA-14
Kimi K3 vs Claude: what tokens actually cost
A token cost audit of Moonshot's Kimi K3 against Claude Fable 5, Opus 5 and Sonnet 5, with the self hosting maths on a 2.8 trillion parameter open weight model, and what the pricing reveals about why Chinese and American labs are running opposite strategies.
Luis Hasanaj · Read → - September 2026ResearchAGA-13
TimesFM 3: forecasting as next token prediction
Google Research released TimesFM 3 on 31 August 2026: a 330 million parameter decoder only model that forecasts time series the way a language model predicts the next word, now with native multivariate forecasting and covariates. What it changes for businesses that forecast demand, and why the licence on the weights matters more than the benchmark scores.
Luis Hasanaj · Read → - September 2026OperationsAGA-19
EU AI Act enforcement: the first month
Enforcement powers under the EU AI Act switched on across all 27 member states on 2 August 2026. A month later the more useful question is not who has been fined but who can actually fine you, and the answer is uneven enough that identical conduct carries different risk depending on the country.
Luis Hasanaj · Read → - September 2026OperationsAGA-12
EU AI Act 2026: what businesses need to know
A practical EU AI Act compliance guide for businesses, current as of September 2026: what already applies after the Digital Omnibus amendments, what lands in December 2027 and August 2028, whether you are a provider or a deployer, what counts as high risk AI, the fines, and a checklist you can run this quarter.
Luis Hasanaj · Read → - August 2026ResearchAGA-11
Read and write: how memory works in the brain, and how we engineer it into AI systems
Memory is not a store but a read/write process: encode, consolidate, retrieve. This paper maps that architecture onto AI-native systems: the context window as working memory, vector and structured stores as long-term memory, embeddings as encoding, and retrieval as recall, and why designing memory first is what separates knowing a fact from having been trained on it.
Luis Hasanaj · Read → - August 2026ResearchAGA-09
Brain blueprints of AI: how neuroscience shaped the architecture of modern intelligent systems
A brief history of intelligence modelled on the brain, from the 1943 artificial neuron to today's transformers and retrieval, and how those biological principles map onto the way AI-native systems are architected now.
Luis Hasanaj · Read → - April 2026ResearchAGA-04
Quantitative signal discovery: a closed-loop multi-agent architecture
A closed-loop architecture where three specialized agents mine candidate alpha signals from market data, translate them into self-correcting Python, and validate them with statistical backtests, refining the search on every pass instead of generating once and hoping.
Luis Hasanaj · Read → - February 2026ResearchAGA-02
Governed autonomy: approval lanes and audit trails in agentic systems
How to give AI agents real autonomy without losing control: a pattern of approval lanes, reversible actions and complete audit trails that lets a business trust software to run part of its operation.
Luis Hasanaj · Read →