SurfacePrint

Typographic Fingerprints
of LLMs

Yuntian Deng

CIFAR meeting, October 14, 2026

The words match. The apostrophes differ.

don't
Straight apostrophe
don’t
Curly apostrophe

People, models and editing software can produce either

Illustrative example

A formatting habit becomes a measurement

Five apostrophes in an illustrative text
’ ’ ’ ' '
3 / 5

60% curly

One of 97 measurements from raw text

Example feature: curly apostrophes divided by all apostrophes. These are illustrative counts.

A small classifier scores the formatting

InputRaw text
Measure97 features
Learned modelClassifier
OutputScore

Higher scores indicate more model-like formatting

No language-model calls when scoring a new text

The classifier learns from labeled human and generated examples

We test new texts from 17 known model sources

Train

Human text + outputs
from 17 AI models

Test

Held-out texts
from those same sources

About 8,000 human and 8,000 generated training examples

Programming   /   Math   /   Explanations   /   Fiction
Human posts from StackExchange and Reddit. All 17 models are represented in training.

AUROC measures how well the scores rank texts

How often does generated text
receive a higher score than human text?

0.5Chance ranking
1.0Perfect ranking
An evaluation over many texts. A single document’s probability of AI authorship requires more evidence.

Formatting strongly separates the benchmark texts

Mean AUROC across the 17 model sources0.5Chance0.60.70.80.91.0PerfectSurfacePrint, rawLearns from labeled examples0.983Fast-DetectGPTTraining-free detector0.950
Both methods use the same held-out test texts. They use different amounts of supervision.

A signal remains after whitespace cleanup

Two variants, each trained on its own version of the text

Raw text
First paragraph.


Second paragraph.
0.983 AUROC
Whitespace standardized
First paragraph.

Second paragraph.
0.889 AUROC
Illustration: repeated blank lines become one. SP-WS also standardizes line endings and trailing whitespace.

Instruction-tuned text is often easier to detect

Separate whitespace-standardized detectors for each condition

Base modelTrained to follow requestsAUROC0.70.80.91.0Llama 20.810 / 0.804Llama 3.10.744 / 0.880OLMo 30.709 / 0.958Qwen 3.50.840 / 0.978Gemma 40.754 / 0.963
Higher AUROC means easier to distinguish from human text. Four of five model families improve.

Transfer to new model sources is uneven

New experiment: train on human text + Claude Sonnet 4 only

Test on model sources unseen during this detector’s training0.00.250.5Chance0.751.0Claude Sonnet 4.6Another model in the Claude family0.969OPT-IML 1.3BA different model family0.301
One SP-WS detector. AUROC below 0.5 means the learned ranking often runs in the wrong direction.

Formatting edits change what we measure

Original text
**A simple answer**

It doesn’t change.
After a formatting edit
A simple answer

It doesn't change.
The message remains while the measured features change
Illustration only. We have not measured the detector’s response to this particular edit.

Practical use needs a measured false-positive rate

How often would
human documents
be wrongly flagged?

A score cutoff and a representative test set make this measurable

We have not yet measured that error rate in the intended application setting.

Formatting offers an
additional detection clue

Its reliability depends on the source and processing of the text

Questions