← All posts

Statistical Attacks That Break Z-Score Watermark Detection and What Actually Survives Them

Substitution and paraphrasing weaken watermarks, but bias inversion breaks them entirely.

November 7, 2026

Cover illustration for “Statistical Attacks That Break Z-Score Watermark Detection and What Actually Survives Them”

KGW watermark detector evasion by z-score works by reducing the excess of green-list tokens that the detector measures. Substitution, paraphrasing, dilution and bias inversion can weaken that signal, while low-entropy generation can weaken it without an attacker. Architectural dispersal keeps a unified single-model signature from forming, but the lack of that signature alone does not prove provenance or regulatory compliance.

How KGW watermarks embed a detectable signal in token sequences

KGW watermarking applies a statistical preference during token generation. At each step, a context-dependent hash partitions the vocabulary into green and red lists, and a positive logit bias δ increases the probability of selecting green tokens. The partition changes with context but is deterministic given the context and key. Detection then checks whether the share of green tokens in the sequence is unusually high.

The detector expresses that excess as a z-score. Its decision reduces to a threshold test on the empirical green-token rate, with sequence length T affecting how strongly the excess registers. Longer texts accumulate more evidence, while short texts are structurally harder to detect. Increasing δ strengthens detection but also increases perplexity, imposing a text-quality cost. In low-entropy settings such as code generation and mathematical reasoning, the next-token distribution concentrates on so few candidates that the bias has little room to change the green/red balance.

How do statistical methods bypass KGW z-score detection by design?

The measurable green-token rate is also the attack surface. A method that lowers that rate sufficiently can move the score below the detection threshold, regardless of whether it changes tokens, rewrites sentences or mixes sources. Lettershred addresses that dependence through token-level multi-model blending rather than a removal pass, dispersing authorship so that one model’s partition does not govern the finished sequence. That distinction separates suppressing an existing signal from preventing a unified one.

Three design properties give an attacker further leverage. The deterministic partition can be targeted if it can be approximated, while tolerance for minor edits can leave room to suppress green tokens without destroying meaning. Public detection APIs can also become feedback oracles that let an attacker test whether a change worked. Length strengthens evidence only when the underlying bias remains coherent. When independent models with different hash partitions supply the output, they break up the statistical fingerprint that a single-model detector expects.

Pang et al.’s NeurIPS 2024 work demonstrates piggyback spoofing, which exploits watermark robustness to place a signal in otherwise non-watermarked text. Enhancing LLM Watermark Resilience Against Both Scrubbing and Spoofing Attacks reports a KGW spoofing z-score of 11.09 using that attack as a baseline. Removal leaves evidence missing, while spoofing plants false evidence, which has more serious consequences for attribution. These are the two attack families, scrubbing a genuine signal and manufacturing a misleading one.

Token substitution and paraphrasing as watermark scrubbing strategies, and where they fall short

Simple substitution attacks swap green tokens for red or neutral ones, while paraphrasing changes more of the sequence. Both can lower the green-token excess that the detector measures. Neither automatically preserves the original meaning, however, and naive query-free methods can exchange detectability for distortion. Early KGW work acknowledged that insertion, substitution and deletion have limited success because the signal is distributed across the full text. Removing enough evidence can thus take so much rewriting that the quality suffers.

Adding non-watermarked material lowers the overall green-token rate without requiring changes to the original watermarked passages. This exploits the global nature of the z-test. It can substantially reduce the score while preserving perplexity because the added text is not distorted merely to suppress green tokens. The marked segments still exist, but their contribution to the combined statistic is diluted.

Fidelity remains the binding constraint for direct scrubbing. A method needs a way to preserve meaning while suppressing the statistical preference, rather than assuming that any fluent replacement will do both. Other architectures, including Lettershred’s blending of independent sources before final editing, aim to avoid that trade-off after generation. They still need to produce coherent text. The limitation of older scrubbing methods is narrower, namely that surface editing alone provides no explicit mechanism for balancing faithful rewriting against signal suppression.

Bias inversion as a theoretically grounded scrubbing attack that solves the fidelity problem

Hwang et al.’s Bias-Inversion Rewriting Attack, or BIRA, targets the generation probabilities used during rewriting. It applies a negative logit bias to a proxy set of likely green tokens, suppressing those choices without requiring the watermark key or the identity of the generating model. The intervention operates below the surface wording. Instead of replacing tokens after selection, it steers the rewriting model away from choices likely to reinforce the watermark.

The BIRA study demonstrates that detection probability decays exponentially in δ, where δ > 0 denotes the reduction in green-token generation probability. This δ describes the attack’s reduction, rather than the positive embedding bias discussed earlier. The study reports over 99% evasion across recent watermarking algorithms while preserving semantic content. On KGW, it reduces a z-score from 6.03 to 0.83, below the standard detection threshold of 4.

BIRA is also query-free with respect to the watermarked model. It works from generated text, making the attack applicable in black-box settings where only the output is available. The Self-Information Rewrite Attack, or SIRA, extends the evidence beyond green-list schemes, reporting near-complete evasion even on EXP, introduced by Aaronson and Kirchner in 2022. EXP has no green list to invert.

What low-entropy collapse reveals about where token-level watermarks are inherently weakest

An adversary is not required for the signal to weaken. Code, structured data and mathematical reasoning often constrain the next token so tightly that a green-list preference cannot substantially change the choice. The generation process still applies the bias. Yet the resulting sequence may accumulate little usable evidence, because the available alternatives are not sufficiently competitive. This makes watermark strength dependent on both the kind of content being generated and the configured bias.

The SSG paper at ACL 2026 formalizes this through “watermark strength,” a token-level measure of the normalized probability shift toward the green set. When high-logit tokens concentrate on one side of a random partition, that shift can approach zero. SSG proposes a logit-balanced partitioning method that raises the lower bound on watermark strength and improves detection on code and math datasets. Random vocabulary partitioning does not guarantee a useful balance of probability between the two sets.

He et al.’s EMNLP 2025 scheme finds the same tension in high-determinacy parts of speech. Low-entropy tokens such as function words support sentence structure, so perturbing them can reduce fidelity without adding much signal. The scheme excludes these tokens from logit perturbation and reports a 24.53% semantic-fidelity improvement over KGW as measured by RC-PPL, while maintaining detectability. A pipeline producing prose together with templates, code snippets and data summaries operates across regions where the same token-level watermark has different strength.

Semantic-level watermarking schemes and what they actually survive

Semantic watermarking moves the signal away from exact token identities. Semantic invariant watermarking bases its logic on embeddings of preceding tokens, so meaning-preserving substitutions need not change the representation that controls the watermark. This raises the cost of simple surface editing. Replacing a word with a synonym may change its token-level partition but leave it in the same semantic neighborhood.

SemStamp applies that principle at the sentence level. It maps candidate sentences into an embedding space divided into watermarked and non-watermarked regions through Locality-Sensitive Hashing. Sentence-level rejection sampling keeps going until it finds a candidate in the watermarked partition. The signal resides in the semantic neighborhood. Consequently, paraphrasing does not reliably remove it merely by replacing the original sequence of tokens.

Google DeepMind’s SynthID-Text instead adjusts logits during generation without incorporating semantic information into those adjustments. Meaning-preserving paraphrasing, copy-paste changes and back-translation can therefore significantly degrade its detectability. The proposed SynGuard hybrid adds semantic information to detection and reports an average 11.1% improvement in F1 score over SynthID-Text. A rewrite that shifts the semantic embedding can still move text across a semantic watermark’s boundary, so these schemes raise attack cost.

Why multi-model pipelines create a provenance gap that watermarks alone cannot close

A robust watermark still answers a narrower question than pipeline provenance. Existing schemes are built for single-model generation and lack a way to trace information as it passes through several agents and model transitions. When model A’s output goes to model B for editing, summarization or augmentation, A’s watermark is not reliably preserved. The handoff breaks the attribution chain. A final detector result therefore cannot establish the complete origin and transformation history of the finished text.

The EU AI Act also requires documentation. Its requirements for documented provenance in high-risk AI systems, with enforcement beginning December 2027, expose the gap between detecting a statistical trace and accounting for a system’s lifecycle. A missing watermark cannot fill that gap. AI Bills of Materials, or AIBOMs, extend CycloneDX with AI-specific provenance, model lineage and disclosure metadata to provide machine-verifiable lifecycle assurance. They address infrastructure and records, rather than making an output watermark do work it was not designed to perform.

The architectural response is to capture provenance, transformations, versions, quality metrics and ownership at each pipeline stage. Those records support compliance and incident response in a way that post-hoc annotation cannot substitute for. Model fingerprinting addresses a related but distinct identification problem through black-box identification backdoors, white-box weight watermarks and model heritage inference. LLMMap, presented at USENIX Security 2025, is one tool in that research area. Identification nevertheless remains an open problem, particularly under fine-tuning, and is not equivalent to a documented chain of transformations.

Why architectural dispersal across models is the only approach that survives scrubbing by construction

BIRA, SIRA and dilution show different ways to disrupt token-level evidence, while low-entropy collapse shows why that evidence can fail to form reliably. Semantic schemes strengthen the signal’s resistance to surface changes, but still encode something an attacker can try to alter. Dispersal takes a different route. Within the approaches considered here, it alone avoids depending on the survival of one model’s watermark through subsequent rewriting.

For KGW-family detection, cross-provider token authorship breaks the premise of a consistent green-list preference across the finished sequence. Independent models contribute different distributions and partitions. This is signal avoidance by construction. It also remains distinct from compliance, which requires the pipeline records discussed above. An absent single-model signature establishes neither human authorship nor a complete account of machine involvement.

Lettershred’s pipeline implements dispersal through parallel model kernels, line-by-line selection by a judge panel and a final enhance pass. The selection and editing stages aim to give text from different sources one voice. The workflow takes in a brief and produces a single on-brand article, so the content team does not have to piece together separate drafts. The architectural claim concerns how authorship is distributed. Its evidence should therefore be judged separately from claims about any individual watermark’s resistance to rewriting.

The enhanced scrubbing- and spoofing-resistance research identified as Arxiv 2507.06274 pursues higher residual z-scores after attacks and lower scores on fabricated watermarks. Lettershred’s reported example instead moves the output z-score from 15.46 to -0.32, below the detection threshold of 4, with 117 of 239 tokens rewritten through cross-provider dispersal. This null detection result establishes the outcome of the reported test alone. The labs ship watermark tools, and content teams deserve an architectural counter that is just as rigorous and is judged by both its statistical results and its provenance records.

AI Watermark Detection

Sources