Entropy and Perplexity as Signals in AI Text Detection
These metrics measure text properties, not authorship.
November 4, 2026

Entropy and perplexity underpin statistical AI text detection, but neither establishes authorship. They measure features of a probability distribution. These limits explain why a second signal cannot ensure reliable detection, and why multi-model authorship changes how the text is made rather than changing words until a detector’s score falls.
What entropy and perplexity actually measure, and why they are not the same thing
Perplexity measures how surprising a sequence of tokens is to a reference model. The model assigns a probability to each observed token given its context, and perplexity combines those probabilities into a measure of overall predictability. Low perplexity means the observed choices were relatively foreseeable. The reference model is therefore part of the measurement: a perplexity score describes the relationship between a text and that model.
Entropy measures how much uncertainty there is about which token will be chosen. When many tokens have similar probabilities, the distribution is flat and entropy is high. When probability mass concentrates on a few candidates, entropy is low. The distinction is between the choices available and the choices actually made. The NLP2CT survey, A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions, describes entropy as an early zero-shot detection signal associated with diversified or unclear content.
These signals are related, but they are not interchangeable. Low perplexity can mean that high-probability tokens were chosen again and again, while high entropy means that probability mass is spread out before a token is chosen. A detector can therefore receive ambiguous evidence from one and stronger evidence from the other. Burstiness adds a different dimension by measuring variation in sentence-level perplexity. Human writing often alternates between predictable and unconventional passages, while model output tends toward greater uniformity.
The Counter Turing Test (CT²) paper also examines burstiness through clustered word choices and broader lexical variation across sentences. Each measure has a distinct object of measurement. Perplexity, entropy, and burstiness describe different facets of the same underlying fact, which is that a language model’s sampling process leaves a statistical fingerprint.
How each signal gets operationalized inside a real detector
A perplexity detector begins by scoring tokens with a reference model. It takes the average of their negative log probabilities and exponentiates the result to get one score for the whole sequence. A calibrated threshold then separates the ranges associated with generated and human text. The CT² paper formalizes a log-probability comparison between human and AI text, making the assumed separation between the two populations explicit.
Burstiness requires a different aggregation of the same underlying token scores. The detector computes perplexity for each sentence, then estimates how much those values differ across the passage. The CT² paper expresses this comparison through the entropy of sentence-wise perplexity differences, with the human side expected to exceed the AI side. Entropy and variance are not identical quantities. Here, both concern dispersion across sentences rather than the average predictability of the whole document.
KGW watermarking operates on a parallel track. Instead of inferring origin from naturally occurring text properties, it embeds a signal during generation. A hash of the previous n−1 tokens determines a green list and a red list, and generation applies a constant bias to green-token logits. Detection then counts green tokens and computes a z-score. The survey places this family at the logits-generation stage, distinguishing it from post-hoc inference.
Recent systems produce sentence- or paragraph-level risk scores and attribute their classifications to features including entropy, burstiness, and semantic fingerprint residuals. Combining signals changes the decision process, but each component still depends on its own measurement and calibration.
Why perplexity-only detection had a structural ceiling from the start
Statistical predictability and machine authorship are different properties. A perplexity-only detector treats their overlap as though it were a dependable boundary, even though predictable text can be human-written. The weakness precedes any deliberate attempt at evasion. It follows from asking a measurement of token likelihood to establish a fact about origin.
Sampling settings expose that mismatch directly. Temperature, top-p sampling, and beam search reshape output distributions without changing the underlying model. Low temperature concentrates choices, while high temperature produces flatter distributions. Neither setting makes the author more or less artificial. A detector that responds strongly to those differences is partly classifying a sampling configuration.
The writing domain introduces a second source of overlap. Perplexity and burstiness are most useful when human writing displays substantial stylistic variation. Structured long-form writing, product documentation, and formal editorial work naturally constrain that variation. Consistency is often part of the assignment. Consequently, the enterprise domains in which detection may be applied can suppress the very human variability on which the classification depends.
Model capability compounds this problem without requiring adversarial behavior. The CT² paper finds that larger LLMs tend to receive higher AI Detectability Index scores, indicating lower detectability than smaller models. More capable generation overlaps more closely with human statistical distributions, limiting the separating signal. The later arms race accelerated a failure that the measurement already made foreseeable.
How the arms race drove perplexity detection into practical obsolescence
When detection methods are published, model developers can also see what they test for. Once a benchmark makes predictable output a liability, it creates a concrete optimization target. The detector becomes part of the feedback loop. Improvements in generation can then erode the statistical separation on which the published result depended.
The relevant change is not simply higher temperature. Crude temperature scaling can increase randomness at the expense of quality, whereas architectural changes can distribute sampling diversity while preserving semantic coherence. A detector calibrated to an earlier distribution can lose usefulness even when the newer text remains fluent and controlled.
The survey identifies out-of-distribution performance as a core challenge. Detectors trained on one generation of models can degrade significantly when applied to another. The CT² benchmark likewise exposes brittleness across multiple detection techniques. Those findings support the arms-race diagnosis without requiring an unsupported accuracy figure for a particular model or year.
Burstiness offers more resistance than raw perplexity to simple surface changes. Its evidence is spread across sentences, so edits in just a few places have a harder time removing the uniform pattern of variation. Yet it is not immune to changes in generation. Architectural entropy pooling distributes sampling diversity across token positions within sentences, altering the regularity that burstiness measures without necessarily reducing output quality. Larger models increasingly match human burstiness, so combining the two signals does not restore a stable authorship boundary.
How KGW watermarking works at the token level and where its z-score signal breaks
KGW replaces inference about natural writing patterns with an embedded signal. For each token, it hashes the previous n−1 tokens to produce a key that deterministically partitions the vocabulary into green and red lists. A constant bias δ is added to green-token logits during sampling. Green tokens should consequently accumulate above their chance baseline. The detector searches for that excess.
The z-score expresses the excess in standardized terms. It subtracts the expected green-token count under the null hypothesis from the observed count, then divides by the standard deviation under that null. A score above the predefined threshold triggers classification as watermarked. The decision remains statistical despite the deliberate embedding. Its strength depends on how much excess evidence the generation process supplies.
A larger δ strengthens that evidence but can degrade text quality. Deployment therefore requires a compromise between output fidelity and watermark robustness. Temperature also plays a part because choices that focus on a few high-confidence tokens can weaken the green-token signal that builds up, while greater randomness can strengthen it. These controls do not operate independently. Settings that favor the desired text can reduce the detector’s separation from its baseline.
Length also affects the available evidence. When the excess green-token rate holds steady, the z-score rises as the sequence gets longer, so short texts are harder to classify reliably. Social copy, headlines, and structured summaries offer less room for the signal to accumulate. The survey describes Entropy-Weighted Detection as a partial response that reweights tokens by their entropy during detection. It keeps KGW’s generation algorithm, so it changes how evidence is counted while the trade-off between signal strength and output fidelity stays in place.
How the BIRA attack exposes the structural vulnerability in any threshold-based detector
The Bias Inversion Rewriting Attack, published in 2025, targets the green-token rate directly. It suppresses green tokens while preserving meaning, reducing the z-score from above the detection threshold to below it. The attack acts on the detector’s decision variable. That is a more specific objective than making the passage sound different.
The BIRA paper states the relevant structural condition precisely. If a detector’s test statistic is a nondecreasing function of the empirical green-token rate, detection is equivalent to thresholding that rate. Under the attack’s conditions, lowering green-token probability by a small amount at each step can bring detection probability toward zero. The pressure must be consistent across the sequence. The mechanism depends on directional control of the distribution and can work without aggressive rewriting.
The CT² paper reports that paraphrasing and high-entropy word replacement can circumvent watermarked text detection, but with varying losses in detection accuracy. It also finds that more robust watermarking versions, including wv2, are harder to defeat through paraphrasing alone and favors high-entropy word replacement in that comparison. Changing wording is not the same operation as suppressing a measured rate. The former may affect the latter without controlling it.
BIRA’s rate-based argument applies directly where the decision rule has the stated monotone form. Perplexity and burstiness can also be pushed across thresholds, but they are not automatically identical rate statistics. Their shared vulnerability is the use of a movable statistical boundary as a proxy for origin. Reactive editing remains fragile because recalibration can change that boundary, motivating generation with the intended distributional properties from the start, or compliance by construction.
Why multi-model architectures are a structural response rather than a workaround
A single-model pipeline supplies repeated evidence from the same generation process. As a passage grows, perplexity estimates stabilize and sentence-level variation becomes easier to measure. A watermark bias that holds steady also has more statistical support as the text gets longer. Additional text is not merely additional content. It gives the detector more opportunities to accumulate evidence of a recurring distribution.
Multi-model authorship changes the source of that evidence. Different providers train with different data and recipes, producing distinct token distributions and lexical preferences. Cross-provider dispersal therefore changes authorship at the distributional level rather than merely substituting words after generation. For watermark detection, the relevant effect is disruption of a consistently favored green-token rate. A detector can still use its hash partition, but text from mixed authorship may not favor that partition’s green tokens all the way through.
This diversity creates a coherence problem that the architecture must also address. A judge panel selects the strongest material line by line, followed by a final enhance pass intended to produce one consistent register. Selection and enhancement separate the distribution of authorship from the presentation of the finished passage. That distinction is central to the architectural claim.
Lettershred’s published benchmarks provide a threshold-referenced example. They report output z-scores dropping from 15.46 to -0.32, below the detection threshold of 4, with a large majority of tokens rewritten through cross-provider dispersal. Those figures directly address the KGW statistic discussed earlier. They support the reported result without establishing universal immunity to detection. Here, compliance by construction means falling below the relevant detector threshold, which does not prove human authorship.
The broader multi-signal case follows from the same design. Mixing providers can alter reference-model perplexity, sentence-level variation, vocabulary entropy, and watermark accumulation together. The response is structural because it changes which distributions generate the text. Its strongest evidence remains a reproducible result against a defined detector.