CLM-8B, Jev, and language model shape

Between 23 and 28 September, four groups who were not coordinating published work that does the same underlying thing. Each one takes a property of the language model that everybody had treated as fixed, and treats it as a design choice instead.

A model that ranks instead of generating. A model whose entire output is one number between 0 and 1. A model that spends more layers on some tokens than others. A model that produces text in blocks rather than left to right.

None of these are ours. We did not write any of these papers and we are not going to write about them as though we did. What follows is a reading of other people's work, with every result attributed and linked, plus the part we do think is ours: the argument that connects them, and the engineering bill that comes with all four.

1. The word for what is happening

Alex Zhang gave it a name on 26 September, in a short essay called Language Model "Shape". His observation is that since ChatGPT the input and output structure of language models has barely moved. It is the autoregressive, decoder-only Transformer, mapping arbitrary text to arbitrary text, and every model worth using has that shape.

So building an agent has meant writing a harness that fits it. His question is whether the reverse is worth trying: design the shape of the model around the harness and the structure of the task.

He is careful about why nobody does this. Decoder-only models are powerful and flexible, architecture choices are worth a great deal of money at scale, and dense next-token prediction is a natural objective over arbitrary text. Frontier labs have every reason to leave the shape alone. His argument is that independent researchers no longer do, because open models and distillation now make it cheap to move a capability into a different shape and find out.

He uses Jev as his worked example, and describes it precisely: a model whose output space is explicitly constrained to the reals between 0 and 1, which lets it be prefill-only when sampling, which is what makes it fast at one class of question. He is also blunt that this is a tradeoff rather than an improvement, and that Jev is "not meaningfully more intelligent than any frontier model."

We wrote about that model a week earlier, from the software side, in TypeSafe's System One. The argument there was that the control flow belongs in your code and the model should be a typed function call. Zhang is making the supply-side version of the same argument, and putting the two together is the thing we think is worth saying:

You cannot design a model around a harness until you have admitted the harness is in charge. Once the model is a dependency rather than the thing driving the loop, its interface is negotiable. Once the interface is negotiable, so is the architecture. The shape question only becomes askable after the orchestration question has been settled, which is why both arguments surfaced in the same month rather than five years ago.

2. Four shapes, four axes

Set out together, the four pieces of work are each changing a different property.

That is not a trend anybody organised. It is four groups independently deciding that the same black box has edges worth pushing on.

2.1 CLM-8B: delete the decoder

The most radical of the four is CLM-8B, released on 23 September by Stanford and NVIDIA under Apache 2.0, with Jacky Kwok at Stanford as project lead.

The architecture is almost aggressively simple. Two small projection heads, one for state and one for action, on top of a frozen Qwen3-8B encoder, trained with a bidirectional InfoNCE loss. Training ran in three stages: roughly 60M Nemotron question-answer pairs for pre-training, roughly 30M synthetic hard negatives for mid-training, and roughly 1M agentic trajectories for post-training.

It does not generate. It embeds a state, embeds the candidate actions, and ranks them by similarity. Picking an action becomes a vector comparison rather than a sequence of forward passes.

The reported numbers, from the authors and from VentureBeat's write-up:

Read that table honestly and it is not a new model beating an old one. CLM wins where it was fine-tuned and loses where it was not. It is materially behind Jev on zero-shot tool calling and it fails four WikiRacing tasks Jev completes. It is faster everywhere.

Which is exactly Zhang's thesis demonstrated rather than contradicted. A different shape buys a different tradeoff. Kwok's own framing of the architectural difference is that CLM's dual-encoder design lets you cache state and action embeddings independently, where Jev principally caches the state. If your action set is stable, you precompute it once and the per-decision cost collapses.

2.2 TaH2: depth as a variable

Improving Test-Time Scaling with Adaptive Looped Transformers (arXiv 2609.35748) comes from Yichen You, Tianyu Fu, Ning Ding and colleagues at Tsinghua and Yale, posted 28 September.

Looped Transformers reuse the same layers for extra iterations, buying depth without buying parameters. The authors' finding is the interesting part: existing looped models often show a steeper accuracy-compute slope than their non-looped baseline and still underperform it at matched compute. More gain per doubling, from a worse starting point.

Their method, TaH2, makes the looping selective. It jointly post-trains the backbone and an iteration decider using lookahead depth supervision, which generates online labels for whether another iteration actually improves that prediction. The extra depth goes to the tokens that benefit. They report improved maths accuracy at matched decoding compute, with transfer to code, question answering and tool use.

The reason this belongs next to CLM is that it changes shape along a completely different axis. CLM changes what comes out. TaH2 changes how much computation each token is worth, and makes that a learned property of the input rather than a constant in the config.

2.3 d-OPD: order as a variable

d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models (arXiv 2609.35362), from Ruitao Liu, Qinghao Hu, Song Han and colleagues at Tsinghua and MIT, also 28 September, changes the order.

Block diffusion models generate blocks sequentially while denoising many tokens in parallel within each block. Rather than train one from scratch, recent work distils a strong autoregressive model into that shape. The authors identify a real mismatch in doing so: the student predicts from a whole partially denoised block, including visible future context, while the autoregressive teacher's target is defined only from the causal prefix. The supervision does not match the information the student actually has.

d-OPD corrects the teacher distribution to account for the future context the student can see. Across Qwen3 models from 0.6B to 8B they report up to 4.0 points on a six benchmark average over OPDLM, with training time down 1.35 to 1.58 times.

It is the most conservative of the four in ambition and the most instructive about method. The bug was not in the architecture. It was in assuming a teacher built for one shape supervises a student of another shape correctly.

3. The bill nobody is itemising

This is the part we would want a client to hear before anyone gets excited, and it is the observation we think is genuinely missing from the coverage.

A ranker cannot return an answer that was not in the set you gave it. The authors are upfront about this: CLM ranks only from provided candidates, and if all of them are wrong, it still ranks them. There is no "none of these" in a similarity score. You get the least wrong option back, with the same confident geometry as a correct one.

So the accuracy ceiling of the system is not set by the model. It is set by whatever produced the candidate list, which is now code you write, test, monitor and maintain. The model got faster and cheaper. The job of being right moved upstream.

That is the same pattern we found reading the GPT-6 Astra system card: the controls that survive are structural, and the work relocates rather than disappearing. It is also the same pattern as the confidence discussion in the System One note, where a flat probability distribution is the only signal that none of your options fit. A ranker without an explicit abstain path has thrown that signal away.

Three more line items on the same invoice:

• Fine-tuning is not optional for the headline numbers. CLM's state-of-the-art results on DeepSWE and Terminal-Bench are the fine-tuned figures. Zero-shot it trails Jev on tool calling. Adopting it means owning a training loop, not just an API key.
• You inherit the encoder. CLM-8B is locked to Qwen3-8B embeddings. That is a dependency on somebody else's representation, with whatever it does and does not encode about your domain.
• A predefined action set is a precondition, not a detail. The efficiency argument depends on caching action embeddings. If your action space is generated fresh each turn, most of the advantage evaporates.

None of that makes the direction wrong. It makes it an engineering decision with a cost column, which is how we argued about model choice in the small language models note and about price in what tokens actually cost. The metric is still cost per completed task, and now it has to include the cost of the candidate generator.

4. What is actually being replaced, and what is not

It would be easy to read four papers in a week and conclude the Transformer is on its way out. That is not what any of this says, and the details argue against it.

CLM-8B is built on a frozen Qwen3-8B encoder. It is a Transformer with the decoder taken off and two heads bolted on. TaH2 loops Transformer layers. d-OPD distils an autoregressive Transformer into a block diffusion model that is also a Transformer. The block underneath is untouched in all four cases.

What is changing is the contract at the edges: what the model is allowed to emit, how much computation a token gets, in what order tokens appear. That is a far more conservative claim than "new architecture", and far more likely to be adopted, because it lets a team keep the thing that works and renegotiate only the interface.

We made the nine-year version of this argument in Attention Is All You Need, nine years on, which audited which of the 2017 decisions survived scale and which were quietly replaced. The pattern there holds here: the durable part of that paper was removing the sequential dependency, not attention itself, and the parts that got replaced were replaced at the edges rather than in the core.

Zhang's own hot take is worth quoting because it is the most specific claim in his essay and the most testable: he does not think decoder-only Transformers are the natural form for processing an agent trajectory, and he argues that context compaction exists precisely because of that mismatch. An agent attends densely to recent observations and sparsely to old history, so a shape that combined a recurrent model over the history with dense attention over the present would auto-compact by construction. He is explicit that this is intuition rather than a result.

If that turns out to be right, it bears directly on what we wrote in Read and write memory and Why the brain keeps two memory systems. Both argued that agent memory wants two stores with different update rates. A fast dense one over the present and a slow compressed one over history is the same structure, proposed as an architecture instead of as a retrieval layer bolted on afterwards. Whether that is convergent insight or coincidence is not settled by anybody yet.

5. What we would test first

If you are weighing any of this for a real system, the order we would go in.

1. Is your action set enumerable and stable? If yes, a ranker is worth benchmarking and the caching argument is real. If your actions are composed fresh each turn, start with the constrained-output shape instead.
2. Measure your candidate generator before the model. Take a sample of real decisions and check how often the correct action is present in the set your code would produce. That number is the ceiling. No ranker improves on it.
3. Build the abstain path first. Decide what your system does when the top candidate wins by a hair. Without a margin threshold you have no equivalent of "I don't know."
4. Price the fine-tuning loop, not the inference. The published wins are fine-tuned wins. Cost the labelled data, the training runs and the evaluation harness before costing tokens.
5. Run the zero-shot comparison on your own tasks. The published table disagrees with itself depending on the benchmark, which means your workload decides, not the average.

6. Where we would push back

Two honest reservations, offered as reservations rather than conclusions.

The first is that "shape" may be doing more work as a framing than as a mechanism. Three of these four results are obtained by keeping a Transformer and changing its interface. That is a real and useful move, but it is closer to good systems engineering than to architectural research, and calling it a new design space risks overselling what is, so far, four sensible constraints applied to one architecture.

The second is evidence weight. The CLM speed figures are the authors' own tests. Jev's latency and cost claims are TypeSafe's. Both arXiv papers went up on 28 September with no independent replication. Four groups converging in a week is a strong signal about direction and a weak one about magnitude, and the numbers in the tables above will move.

We have not run CLM-8B, Jev, TaH2 or d-OPD against our own workloads. Everything here is a reading of published material, most of it days old, some of it published by parties with a commercial interest in the result. We will test what we can and write up what we find, including if it contradicts this.

If you are deciding what shape of model belongs inside a system you are building, or whether a ranker can replace a generator in a path you already run, talk to us, or see how we build systems for businesses.

7. Sources

All of the work below belongs to the researchers and organisations named. Links go to the originals.

1. Alex L. Zhang, Language Model "Shape", 26 September 2026. The framing this note is built around, and the source of the Jev example and the agent-trajectory argument in section 4.
2. Stanford and NVIDIA, CLM-v0.1-8B model card, released 23 September 2026 under Apache 2.0. Architecture, training stages, benchmark figures and the stated limitations. Code at github.com/Contrastive-LM/CLM.
3. VentureBeat, Stanford and Nvidia's open CLM-8B caches reusable agent actions and runs up to 9x faster than Jev in tests, the head-to-head figures against Jev and the quotes from project lead Jacky Kwok.
4. Yichen You, Tianyu Fu, Ning Ding et al. (Tsinghua, Yale), Improving Test-Time Scaling with Adaptive Looped Transformers, arXiv 2609.35748, 28 September 2026. TaH2, the iteration decider and lookahead depth supervision.
5. Ruitao Liu, Qinghao Hu, Song Han et al. (Tsinghua, MIT), d-OPD: Future-Aware On-Policy Distillation for Block Diffusion Language Models, arXiv 2609.35362, 28 September 2026. The teacher-student information mismatch and the Qwen3 results.
6. TypeSafe AI, Introducing System One models and Jev, 28 September 2026, and the documentation. The RLCD objective, the prefill-only design and the vendor latency and cost figures.
7. Shunyu Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, arXiv 2210.03629, the trajectory pattern Zhang's agent argument is stated against.
8. Recursive Language Models, arXiv 2512.24601, the harness class Zhang cites as a natural fit for constrained-output models.

This note sits in our Research track, alongside the foundation models, agentic and system design threads. The full library is at Research.