Why the Z-Score Is the Weakest Link in KGW Watermark Detection
Paraphrase and multi-model pipelines erase KGW's green-token excess.
November 14, 2026

KGW watermark detector evasion by z-score exploits a limitation of the test itself. The detector measures excess green-token frequency across a sequence. Paraphrase and independent multi-model generation can disperse that excess toward the unwatermarked baseline, making the z-score a weak link in provenance detection.
How KGW embeds a watermark at the token level
KGW embeds its watermark while the model generates text. At each decoding step, a hash of the preceding token partitions the vocabulary into a green set and a red set. Green-token logits receive a fixed additive boost, δ, which shifts the sampling distribution toward those tokens. Because the partition is context-dependent and secret, a reader cannot identify which words received that statistical preference.
Detection works backward from the resulting token sequence. It counts eligible green tokens and applies a one-sided z-test, subtracting the expected green-token count from the observed count and dividing the difference by the standard deviation under the null hypothesis. If the resulting z-score exceeds a preset threshold, ẑ, the detector classifies the text as watermarked. No individual token establishes that result on its own. The detection claim rests on the accumulated excess across the sequence.
Why the z-score is a statistical aggregate, not a measure of intent
A green token is not proof of watermarked generation. An unwatermarked process can select the same token, so the detector must ask whether green tokens occur unusually often across the document. Aggregation is therefore a consequence of the mechanism. The test identifies a distributional pattern rather than a record of which choices the watermark influenced.
The calculation uses three quantities, which are the total eligible token count T, the observed green-token count |S|_G, and the expected green fraction γ under the null hypothesis. The completed text fixes the first two, while the detector relies on its assumed baseline for the third. For a fixed δ, longer sequences can accumulate more signal. Detection confidence is thus entangled with both document length and the strength of the generation-time bias. A stronger score does not establish stronger intent.
Pan et al.’s Tsinghua/ICLR 2025 work shows that a mismatch between the actual green ratio and the preset green ratio γ₀ can cause fixed-threshold z-score detection to fail. That finding exposes a calibration dependency. The detector compares probabilistic output with an expected ratio it cannot independently verify from the score alone. Its conclusion combines variability in generation with a threshold decision about that variability. Changing the threshold does not turn those two uncertainties into direct evidence of origin.
Three structural conditions that collapse the z-score before any attack
Short text creates the first structural constraint because it offers fewer eligible tokens over which the green excess can accumulate. This affects the text that businesses use every day as well as unusual test cases. Short-form content cannot supply the same statistical evidence as a long document generated with the same bias. A threshold validated on longer sequences therefore does not automatically transfer to shorter ones. Length changes the evidence available to the test.
Low-entropy content creates a different constraint because many token choices are already tightly determined by the intended meaning. Documentation, templates and legal drafts face this limit. Li et al.’s EMNLP 2025 study identifies differences in determinacy across parts of speech that KGW’s indiscriminate sampling perturbations overlook. Where semantics already constrain the next token, forcing a green preference conflicts with fidelity. These are central content-team workloads.
High-temperature sampling adds the third constraint because it spreads token choices across a wider distribution. The accumulated green excess can then shrink relative to the variation represented in the z-score denominator. Saifur Rahman Tamim and colleagues’ AIES 2026 forensic readiness preprint measured a 70% baseline false-negative rate for KGW before any attack. That result holds for the conditions the study tested and cannot be assumed for every KGW deployment, but it shows that even cooperative generation can fail to yield reliable detection.
The quality–detectability trade-off that makes the z-score harder to fix than to escape
Increasing δ is the direct lever for strengthening KGW’s generation-time bias, but it also widens the gap between the model’s natural distribution and the distribution used to select tokens. For content teams, a stronger statistical signal can come at the cost of keeping the same voice. A higher green-token preference changes the process that produces the writing.
The Li study documents the semantic cost of indiscriminate perturbation. Its parts-of-speech-guided framework achieved a 24.53% improvement in semantic fidelity, measured by RC-PPL, over baseline KGW by limiting which tokens were perturbed, supporting selective intervention. It does not establish that one parameter setting can maximize both fidelity and robustness. No known KGW variant in the evidence presented resolves both objectives at once.
This trade-off makes it harder to fix the parameters than to disperse the signal. A pipeline that raises δ to preserve detection also applies greater pressure to its token choices, potentially undermining brand consistency throughout its output. That operational cost is distinct from the detector’s benchmark performance. A reliable classification rate would not, by itself, establish acceptable writing quality.
What paraphrase attacks reveal about the z-score's dependence on exact token identity
Meaning-preserving paraphrase tests whether the signal survives ordinary changes in wording. Replacing tokens changes the observed green count and, because the partition depends on preceding tokens, can also change the context against which subsequent choices are classified. When that count moves toward its null-hypothesis expectation, the z-score loses the excess needed for detection. Semantic identity and token identity are different properties.
The forensic readiness study reported 846 valid paraphrase runs across the methods it tested. Every initially detected KGW text lost its detectable watermark after meaning-preserving paraphrase, producing 100% conditional removal for KGW. “Conditional” is essential to interpreting that figure. It describes removal among texts detected before paraphrasing.
The study chose paraphrase because it is legally realistic and difficult to dismiss as evidence tampering when meaning survives. The study casts doubt on the claim that losing a watermark must mean someone set out to destroy substantive evidence, while leaving admissibility in any particular dispute unresolved. An ordinary editor or second model can alter the exact feature the detector counts without changing the content’s message. Consequently, watermark absence cannot distinguish unwatermarked authorship from watermarked text that has passed through meaning-preserving editing.
How multi-model pipelines dissolve the z-score by construction
Multi-model generation changes where the counted tokens come from, going beyond changes to their wording afterward. A second independent model need not follow the first model’s green-red partition. Relative to the original detector, its choices can therefore contribute tokens without the original green preference. As independent contributions disperse that preference, the aggregate moves toward the unwatermarked baseline. This is a consequence of the test’s design.
The distinction is between dispersing the signal structurally and guaranteeing a particular threshold result. Genuine independence removes a reason for green-token excess, but merely using several models does not prove that the final count will fall below ẑ. The relevant conditions include their green-list assignments and their contributions to the output. Where those contributions produce the null-like distribution the detector expects from unwatermarked text, the z-test has no separate mechanism for reconstructing the multi-model process. Its statistic still describes only the final sequence.
Lettershred applies this architectural approach by mixing output from independent models so that no single green list dominates the resulting tokens. Its described pipeline routes a brief through a dozen models in parallel and uses a judge panel to select output line by line. Token authorship becomes the variable being changed. The pipeline aims to keep one voice without keeping a single model’s repeated statistical preference. That makes it a concrete example of cross-provider dispersal rather than a proposed adjustment to KGW’s detector.
In Lettershred’s self-reported benchmark, the z-score fell from 15.46 to -0.32 against a KGW detector with a threshold of 4, with the majority of tokens rewritten through cross-provider dispersal. Those figures describe the tested configuration. They do not establish the same outcome for every detector, model combination or document length. Evaluating the result still requires the named detector, before-and-after scores and token-level change count, even though it is consistent with the mechanism.
Why the stronger objections to z-score reform do not resolve the structural problem
KGW’s operational simplicity is a genuine advantage. It requires no model retraining, no key exchange and no access to model weights for detection. Those properties make deployment easier, but they do not change what the detector observes. The strongest case for KGW is therefore practical rather than evidentiary.
Other methods go beyond the same single-aggregate approach, but come with costs of their own. Semantic-level watermarks such as SemStamp and k-SemStamp, cryptographic watermarks, and behavioral fingerprinting involve latency, infrastructure or quality trade-offs that standard content pipelines may struggle to absorb. Behavioral fingerprinting research identified as arXiv:2509.04504 develops multi-faceted profiles of cognitive and interactive styles. That richer signal requires a diagnostic prompt suite and an automated evaluation pipeline. It is not a lightweight replacement for counting green tokens.
These costs prevent an easy substitution argument. Resistance to model modifications remains an open problem, and the research described here does not provide a drop-in replacement that preserves KGW’s simplicity while resolving its structural fragility. Nor can the paraphrase findings be dismissed solely by labeling rewriting as tampering.
For content teams, the lesson is not to wait for a stronger score to make a single-model watermark into a complete compliance strategy. Cross-provider generation, including the approach used by Lettershred, already changes the authorship structure that the aggregate assumes. It shifts intervention from δ to signal concentration at the pipeline level. However, dispersing a watermark does not itself demonstrate regulatory compliance. The architectural response must be evaluated separately from any claim that a low score proves acceptable provenance.
What content teams and compliance officers should actually verify before trusting a watermark signal
The first verification is the detector and its measured behavior before and after rewriting. A useful claim names the detector and supplies both z-scores. Removal benchmarks also need the token-level change count, because a final below-threshold result alone does not establish what the pipeline changed. Vendor results that favor a method and research findings that criticize it must meet the same requirements. The tested configuration must remain visible in the claim.
The second verification is the content domain. A result validated on long narrative text does not automatically transfer to short, low-entropy or templated enterprise output. Length and token determinacy affect the evidence available to the test. Teams should therefore match the evaluation material to the documents for which they intend to rely on detection. A benchmark’s relevance depends on that match.
The third verification is genuine provider diversity. Two endpoints from the same provider may share green-list logic, so endpoint count is not proof of independent token contributions. Multi-model dispersal depends on the independence described in the mechanism. A pipeline should not receive architectural credit simply because several generation calls appear in its workflow. Teams must verify whether one watermark preference still dominates the final sequence.
The fourth verification is the evidentiary standard. The 2026 forensic readiness evaluation gave KGW a Forensic Readiness Score of 37/60 under Daubert admissibility criteria and rated it NOT FORENSIC READY. That finding challenges reliance on z-score watermarking alone for EU AI Act provenance requirements. A machine-learning benchmark and a legal sufficiency judgment are not interchangeable. Detecting a watermark or removing it successfully still leaves compliance unproven.
The sound architectural principle is to build compliance into the generation pipeline. When authorship is spread across providers, it can produce the null-like distribution that KGW tests for, so the detector has no evidence of a single green-list preference. Provenance evaluation must account for how the text was produced, because an aggregate over its final tokens cannot reconstruct that process.
Sources
- Watermarking with Low-Entropy POS-Guided Token ...
- AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation
- Published as a conference paper at ICLR 2025 CAN WATERMARKS BE USED
- Towards Safe and Efficient Low-Entropy LLM Watermarking
- A Linguistics-Aware LLM Watermarking via Syntactic Predictability