← All posts

How Language Models Leave Statistical Fingerprints in Text

Watermarks are measurable but fragile, and paraphrase erases them.

October 28, 2026

Cover illustration for “How Language Models Leave Statistical Fingerprints in Text”

Language models leave statistical fingerprints because their token choices reflect learned probability biases. Deliberate watermarks make those biases measurable, but they are only one layer of model identity. Understanding how these signals arise, how detection works and where it fails explains the central provenance problem. A detectable pattern is not necessarily durable evidence, and removing a watermark does not erase every trace of the model that produced the text.

Why every language model leaves a mark in its output, even without trying to

A model reveals its preferences through the words it selects. Each token is drawn from a probability distribution that depends on the model’s weights, training data and decoding configuration. Over hundreds of choices, those preferences leave statistical patterns that can tell one model’s output from another’s. The signature is therefore not something added after generation. It emerges from generation itself.

This distributed pattern explains why changing a few suspicious words cannot reliably remove a model’s fingerprint. The remaining choices still reflect the process that produced them. However, this passive residue differs from deliberate watermarking, in which a developer actively modifies token probabilities to embed a signal. Natural Language Fingerprints, or NLFs, arise without that intervention. Both can support attribution, but they require different explanations of what the detector measures and what its result establishes.

How the KGW mechanism turns token selection into a measurable signal

Kirchenbauer et al. (2023) introduced the KGW framework, which adds a deliberate watermark as each token is chosen. The mechanism hashes the previous token with a secret watermark key to divide the vocabulary into a green list and a red list. The partition changes with context. Consequently, a token is not permanently green or red, and the watermark cannot be located by searching for a fixed set of words.

Before sampling, the mechanism adds a positive bias, δ, to the logits of green-list tokens and leaves red-token logits as they are. Logits are the scores that the softmax converts into selection probabilities. Higher green scores make green tokens more likely to be chosen. It does not require every generated token to be green. As sampling repeats, it produces more green tokens than expected by chance, and a detector can measure that excess across the text.

Detection uses the z-test Z_K(T) = (N_g − γn) / √(γ(1−γ)n). Here, N_g is the observed green-token count, γ is the expected green fraction under the null hypothesis, and n is the token count. The numerator measures the excess over chance, while the denominator puts that excess on a standard-deviation scale. A score at or above z₀ = 4 signals watermark presence. In plain terms, the test asks whether the text contains more green tokens than chance would plausibly produce.

Wang et al.’s MorphMark formalizes the quality–detectability trade-off in this design. A larger δ strengthens the signal by shifting more probability toward green tokens, but it also pushes generation further from the model’s natural preferences. A smaller δ preserves those preferences more closely while weakening detection.

Why the green-list bias is structural, not incidental — and what that means for text quality

The conflict between detectability and quality follows from the probability distribution itself. A logit-bias watermark becomes detectable by changing how often particular tokens appear. That change is also a distortion of the model’s learned distribution. Its consequences depend on the surrounding text. There is no independent control that increases the green-token excess without altering selection probabilities.

MorphMark identifies the cumulative probability of green-list tokens as the key variable governing that distortion. When the green list contains tokens the model already favors, an added bias can produce a signal at little quality cost. When it contains tokens the model is unlikely to choose, the same pressure pushes it toward less natural choices. A fixed δ stays the same in both cases.

For content teams, fluent prose is therefore not evidence that a watermark is weak. The green list may simply align with the model’s natural preferences in that passage. N-gram distributions, embedding space clustering and output probability profiles can retain model-identifying information without any deliberate watermark.

How detection actually works in practice, and where it already struggles before any attack

A detector needs enough of the original token stream for the green-token excess to register. Short texts offer fewer observations, while code, structured data and formulaic prose constrain the choices available during generation. Repetitive phrasing and other low-entropy conditions further restrict the places where bias can express itself. A watermark can exist without being detected. The embedding mechanism does not, by itself, establish the reliability of the resulting evidence.

Tamim et al. (2026) found high false-negative rates before any adversarial intervention across all three methods they tested. Unigram missed 83% of genuinely watermarked texts, while KGW and SynthID also missed the majority under the evaluation’s clean conditions. These failures preceded any attempt to remove the signal. A negative result may reflect ordinary detector limitations.

Hong et al. (2026) identify another source of unreliability in statistically disruptive “signature” tokens. Their presence suppresses detection in weak-signal and low-entropy settings. The researchers’ signature filtering method takes those tokens out before the test, which lets it detect far more watermarks under the same conditions. Detection can therefore improve without changing how the watermark is embedded. However, that improvement also shows that the baseline test needs augmentation to approach reliability.

The threshold z₀ = 4 is calibrated to control false positives, including human text incorrectly flagged as watermarked. That protection also means weaker genuine signals fall below the threshold. Forensic interpretation must account for the cases the detector misses as well as those it flags.

Why paraphrase destroys the watermark signal almost every time

Meaning-preserving paraphrase targets the exact feature on which KGW depends. The watermark resides in token identities and their context-dependent green-list assignments. Rewriting replaces that sequence with another sequence. The detector can assign the replacement tokens to green and red lists, but the original selection bias gives those replacements no reason to fall disproportionately on the green list.

The Tamim evaluation tested meaning-preserving paraphrase across 846 valid runs. Every initially detected KGW text lost its detectable watermark after paraphrasing, yielding 100% conditional removal. Unigram also showed complete removal, while SynthID reached near-total removal. The denominator is crucial to interpreting this result. Conditional removal counts only texts detected before paraphrasing, so texts the detector had already missed do not count as successful removals.

This failure mode requires no change in meaning. It overlaps with the rewriting that editors routinely perform, which makes it difficult to treat an intact watermark as a durable property of the content. The experiments establish that vulnerability under their tested conditions, rather than proving that every possible paraphrase defeats every detector. Nevertheless, the mechanism explains the result directly.

Spoofing can cause text to be accepted as watermarked without having come from the watermarked model. Zhao et al. (ICLR 2025) identify a further instability involving differences between the actual green ratio and the preset partitioning ratio. With a fixed detection threshold, a watermark may register under some hash keys but not others, independently of attack.

Diagram: Paraphrase Removes What Detectors Already Struggle to Find. Visualizes: Show the baseline false-negative rates for three watermarking methods before any attack, then the conditional removal rates after meaning-preserving paraphrase.

What the forensic fragility of watermarks means for the regulations that assume they work

EU AI Act Article 50 and California SB 942 require disclosure in terms that call for detection that holds up over time. The EU requirement describes markings as “sufficiently reliable, interoperable, effective and robust as far as this is technically feasible”. California’s disclosure requirement uses the phrase “permanent or extraordinarily difficult to remove”. These are performance requirements. Their practical force depends on what remains detectable after content leaves the generation system.

The Tamim evaluation assesses KGW, Unigram and SynthID-Text against Daubert admissibility criteria and the NIST SP 800-86 digital forensic process. None of the three satisfies more than two of five Daubert factors. The same evaluation reports a 70% false-negative rate for KGW before adversarial intervention. These results expose a substantial gap between the disclosure obligation and the forensic readiness of the tested methods. They do not establish that inserting a watermark supplies reliable evidence by itself.

The evidentiary weakness operates in both directions. A prosecutor relying on a positive watermark result faces a signal that ordinary paraphrase can eliminate, as well as the separate possibility of spoofing. A defense relying on a negative result cannot establish human authorship when genuine watermarks already go undetected so often. Compliance obligations remain despite that uncertainty. Content teams must manage provenance risk with existing tools without treating their outputs as conclusive proof.

How fingerprinting extends beyond watermarks into model weights and behavioral traces

Removing a watermark addresses only the injected signal. Traces of how a model behaves can also show which model it is, including lasting token-preference bitstrings and n-gram distributions. Active fingerprinting uses optimized probe queries to elicit model-specific responses even when only black-box access is available. This changes the object of examination. The method tests how a model responds to carefully chosen inputs, moving beyond the search for a keyed excess in a passage.

Hu et al.’s LLMPrint framework (2025) optimizes prompt suffixes to induce model-unique token-preference bitstrings. It reports true positive rates up to 0.96 with near-zero false positive rates. Those results demonstrate attribution through behavior without relying on an embedded text watermark. However, probe-based model identification is not the same task as detecting a watermark in an arbitrary passage. Its performance does not make the two forms of evidence interchangeable.

Layerwise statistics of Q/K/V/O matrices can remain identifiable after heavy continued training or architectural modification, with layer correlations above 0.8 identifying shared ancestry. Attribution may remain possible through other traces, depending on the access and tools available.

Why single-model pipelines are structurally exposed and what architectural diversity achieves

A single-model pipeline keeps drawing on one model’s learned preferences, even as its next-token distribution changes with context. Post-hoc editing does not necessarily diversify that source. If the same model generates the replacements, its passive statistical identity can persist even when the original watermark sequence has been broken. The architectural response is to distribute authorship across models.

For keyed watermarks, that approach follows from what the z-test counts. When tokens come from different models whose keys give them different green-list assignments, they may not add to the same keyed excess. Cross-provider dispersal can therefore weaken the accumulation that one detector expects to find. The extent of that weakening remains an empirical question. Multiple models do not, by their mere presence, prove that every watermark or behavioral fingerprint has disappeared.

Lettershred implements this principle by running a dozen kernels in parallel and using a judge panel to select the best output line by line. The text it puts together comes from multiple models across multiple providers. Lettershred reports a z-score drop from 15.46 to -0.32 against a detection threshold of 4, with 177 of 240 tokens rewritten through cross-provider dispersal. That result demonstrates suppression of the measured signal in the reported case. It does not establish immunity to every attribution method discussed above.

The judge panel also tackles the main concern editors have with multi-model authorship, that the voice can change from one part to the next. It selects tokens and passages for quality and coherence as well as statistical dispersal. Hong’s filtering results show why the mix of tokens affects detection, since taking out disruptive tokens can strengthen a signal that an unfiltered test misses. The architectural conclusion is therefore specific. Diverse authorship can prevent one source’s watermark from dominating the measured sequence, while broader provenance claims still require evidence beyond a single z-score.

LLM Fingerprinting

Sources