Verifier errors inside a GRPO group are not independent

This paper reads Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts by Esther Xin, an independent researcher. The work, the corpus and every figure and table below are hers. What is ours is the reading: which number carries the argument, which one does not, and what follows for anyone running group-relative RL. We have not reproduced any of it.

Group Relative Policy Optimization samples k completions for one prompt, scores each with an automatic verifier, and builds the policy gradient from the within-group relative advantage. The advantage is centred on the group mean rather than on a learned baseline, and that choice is usually defended as noise robustness: averaging over k samples should wash out any single verifier mistake.

That defence is a statistical claim. It requires the k verifier errors to be independent. Almost nobody writes that down, and this paper is the first to go and measure it on real rollouts.

1. The number

On 24,998 complete groups of eight completions from Qwen2.5-1.5B-Instruct, the pooled within-group verifier-error correlation is 0.530, with a 95 per cent bootstrap confidence interval of [0.500, 0.560].

Run that through the Kish design effect, DEFF = 1 + (k̄ - 1)ρ, and an eight-completion group carries the statistical information of 1.70 independent samples. Not eight. Fewer than two.

The estimate is not a small-sample artefact. It was tracked during collection and barely moved: 0.523 at n=2,000 groups, 0.523 at n=4,000, 0.530 at the full 24,998.

The corpus construction is unusually careful. Fifty thousand prompts from GSM8K train (7,473), MATH train across all seven subjects (7,500), and a DeepMath-103K top-up (35,027). Eight completions per prompt at temperature 0.9, top-p 0.95, of which 24,998 groups came back complete, for 199,984 traces. Each request asks for all eight completions at once, so group membership is never reconstructed after the fact, and a group returning fewer than eight is dropped and counted rather than backfilled. That is the detail that makes the ICC mean anything.

2. The mechanism, which is the actual contribution

The correlation is not mysterious. It falls out of two facts that were already published separately.

First, verifier error is category-concentrated. Xin's earlier audit, Where the Verifier Fails, ran certified metamorphic rewrites of gold answers through four verifier implementations and found self-validation varying by 41.3 points on identical inputs, with whitespace and punctuation accounting for 93.0 per cent of in-contract failures for the default LaTeX configuration. Error clusters by answer form, not uniformly across parsing modes.

Second, a GRPO group is repeated sampling of one prompt by one policy. The eight rollouts are therefore far more likely to share an answer form than two arbitrary answers would be.

Put those together and the correlation is forced. Shared prompt means shared answer form, shared answer form means shared error mode, and the verifier makes the same mistake eight times.

3. The split, and the inversion inside it

Breaking the ICC down by primary answer category is where the paper stops being a single number and becomes a finding. The split is clean.

Categories needing only formatting normalisation sit at zero. Categories needing structural or semantic parsing run from 0.38 to 0.69.

This is the inverse of Xin's own earlier result, and it is the part that earns the paper its place. The categories consuming the most aggregate false-negative budget, whitespace and punctuation, are the categories whose errors are least correlated within a group. The categories that barely register in an aggregate error rate are the ones that poison a training group.

The reason is mechanical once stated. A trailing newline is a property of one sampled completion, so it varies sample to sample. Whether the answer is a radical is a property of the problem, so all eight rollouts carry it. One error mode is idiosyncratic noise, the other is a fixed property of the prompt wearing noise as a costume. So a verifier audit that ranks failure modes by aggregate rate ranks them in close to the wrong order for training.

4. What the paper refuses to claim

This is the part that tells you the author is honest, and the part that caps what the result supports.

The reported ρ is a marginal correlation, pooled across prompts of varying difficulty. Pooling over heterogeneous per-prompt error rates induces positive marginal correlation on its own, with no correlated verifier error required: easy prompts contribute uniformly-correct groups, hard prompts contribute uniformly-wrong ones. The paper says so explicitly and declines to claim that ρ > 0 refutes conditional independence given the prompt. Separating the two is left to future work.

The same discipline runs through the degeneracy result, which is the one a casual reader will misquote. A group is degenerate when every member gets the same reward, so sd(r) = 0 and the advantage is zero for all eight.

Observed degeneracy runs roughly 6.3 to 6.5 times the homogeneous-difficulty reference p^k + (1 - p)^k from Gradient starvation in binary-reward GRPO. The paper does not attribute that gap to correlated verifier error, because the reference assumes a single global pass rate and ordinary difficulty heterogeneity inflates degeneracy above it all by itself. It also notes that zero advantage is not zero gradient, since GRPO objectives normally keep a KL term to a reference policy that survives when the reward-derived advantage vanishes.

A paper that wanted a headline would have claimed the 6.5x. This one does not.

5. Where the gain actually comes from

Two results in the paper are weaker than they look, and the author flags both.

The sign-flip rates are tiny. Replaying the advantage across the four rule-based verifier configurations (strict exact match, loose normalised match, numeric at 1e-4 relative tolerance, and flex extending that to percent, fraction and set forms), the worst pair disagrees on the sign of the advantage for 0.0018 of traces, corrupting 0.0083 of groups. numeric against flex corrupts 0.0005. The four rule configurations mostly agree on clean boxed answers, so the choice between them is not the problem.

The router's accuracy is a coverage artefact. A category-conditioned router, picking the single best verifier per category against 5,556 human-adjudicated labels, reports 97.1 per cent accuracy on the 3,088 traces it covers, which is 55.6 per cent of them. Strict gets 75.9 per cent at full coverage, loose 89.5 per cent at full coverage, numeric 98.2 per cent on the 1,717 traces it can parse, flex 97.4 per cent on 2,025, and CompassVerifier-3B 94.9 per cent on 2,367. None of these is matched on coverage, and the router's per-category choice was fit on the same labelled set it is scored on with no held-out split. The paper puts both caveats in the table caption rather than a footnote. The honest reading is a feasibility demonstration, not a win.

So the load-bearing number in this paper is ρ = 0.530 and the category table under it. Everything else is context.

6. Three positions of ours

One. The hedge in section 4 is scientifically correct and operationally irrelevant. Whether the dependence comes from correlated verifier error or from difficulty heterogeneity, the group-mean baseline sees the marginal dependence, and the normalisation is biased either way. The causal decomposition matters enormously for deciding what to fix. It matters not at all for deciding whether something is broken. Readers will treat the hedge as a reason to wait for better evidence. It is not.

Two. Raising k is close to useless here, and that follows from the paper's own formula. Effective sample size under an exchangeable-error model is k / (1 + (k-1)ρ), which converges to 1/ρ as k grows. At ρ = 0.530 that ceiling is 1.89 effective samples, for any k. Going from eight rollouts to sixteen moves effective samples from 1.70 to about 1.79, for double the sampling compute. The author does not state this, but it is arithmetic on her number, and if it holds then a large slice of the GRPO group-size tuning the field does is buying a rounding error. The counterargument is that larger k buys other things, pass@k coverage and a better chance of a mixed group, and those are real. It is the noise-attenuation argument for larger k that dies.

Three. The perfect annotator agreement should have blocked the router section. Raw agreement, Cohen's kappa and Krippendorff's alpha all came back at 1.00 on the 200-trace double-annotated overlap. The paper reports it as measured and flags it as higher than expected for a task containing genuinely ambiguous equivalence judgments. We would go further. On a label set that deliberately includes symbolic_other, where the whole question is whether two differently-formatted expressions are the same, kappa = 1.00 is evidence the annotators shared a convention, not evidence the labels are right. That weakens the router numbers. It does not touch ρ, which is computed from rule-verifier verdicts against gold answers and needs no human labels at all. The author says so, which is why the main result survives.

7. What this does not establish

• One policy family, one domain. Qwen2.5 on mathematics. Nothing here says anything about code verifiers, which have a different error taxonomy entirely.
• The scaling claim is a single data point. Qwen2.5-7B-AWQ shows ρ = 0.657 and n_eff = 1.43, but on a partial corpus of 6,500 groups, 13 of 50 planned shards, with AWQ quantization present on the 7B run and absent on the 1.5B run, so model scale and quantization are confounded. No bootstrap CI was computed. Its mixed-group rate of 0.298 misses the 0.30 threshold calibrated on the 1.5B pilot.
• No training run. The claim is about the statistical structure of verifier error and its gradient consequence. No policy was trained to convergence under any correction, so the cost in final model quality is unmeasured.
• The taxonomy is borrowed. The 17 categories were validated on synthetic certified rewrites, not on natural rollouts. Identical extraction code was used on both corpora, which mitigates but does not remove the risk that a boundary calibrated on synthetic data partitions real rollouts differently.
• The n ≥ 100 threshold is post hoc. It was chosen after watching large_numeric move from ρ = 0.0 at n=29 groups to 0.660 at n=243. Reasonable, and not pre-registered.
• Single author, no independent replication. Code and data are released, which is the right response, but nobody else has run it.

8. Why this matters beyond RLVR

Existing noise-correction methods for GRPO reduce verifier quality to one or two population-level scalars. Noise-corrected GRPO models reward corruption as Bernoulli noise and derives an unbiased gradient from it. Others treat the verifier as a stochastic channel with a false-positive and a false-negative rate. Each is internally correct conditional on independence. None was calibrated against a population with ρ = 0.53 inside every group.

The axis matters too. The LLM-judge-panel literature, Nine judges, two effective votes, measures correlation across judges scoring one item, and the remedy there is at least conceptually available: find less correlated judges. This paper measures correlation within one verifier across the members of one training group. Adding verifiers to a majority vote does not touch it, because the unit that needs decorrelating is the group, not the panel.

We keep running into that shape. A system inherits an assumption along with the statistic it borrowed, nobody restates the assumption at the new site, and it fails there. The measurement version of this argument is in AGI talk measures the wrong axis, the footnote version in GPT-6 Astra: what the footnotes say. This is the statistical version, and the cleanest, because here the unwritten assumption has a number attached.

9. Sources

1. Esther Xin, Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts. Every figure, table, quotation and number above is from this paper unless stated otherwise. Code and data at ethxin0011/rlvr_group_correlation.
2. Esther Xin, Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR, the earlier audit this paper takes its category taxonomy from, and the source of the 41.3-point and 93.0 per cent figures in section 2.
3. Shao et al., DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, where GRPO is introduced.
4. Nie et al., Gradient starvation in binary-reward GRPO, the homogeneous-difficulty degeneracy reference used in section 4.
5. Yang et al., Your Group-Relative Advantage Is Biased, which proves the group-relative estimator is biased conditional on non-degeneracy.
6. El Mansouri et al., Noise-corrected GRPO: From Noisy Rewards to Unbiased Gradients, one of the independence-assuming corrections section 8 discusses.
7. Guneet Kohli, Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels, the cross-judge result this paper borrows the Kish framing from and distinguishes itself against.
8. Our own When the verifier moves inside the model, on what disappears when a correctness check becomes a training-time object, and Attention Is All You Need, nine years on, for the architecture all of this trains.

What we have not verified: anything. We have not run the released code, resampled the corpus, recomputed the ICC, or checked the Kish arithmetic against the raw verdict matrix. We have not reviewed the human labels, which is where we would start given the kappa of 1.00. The 1.89 effective-sample ceiling in section 6 is our derivation from the paper's own design-effect formula and its own ρ, not a result the author reports, and we have not confirmed the exchangeable-error model it assumes holds well enough at larger k for the ceiling to bind in practice. The positions in section 6 are ours to defend and yours to argue with.

This note sits in our Research track, alongside the foundation models and benchmark threads. The full library is at Research.