Research talk ·
SurfacePrint
Typographic Fingerprints of LLMs
Small formatting choices carry information about how text was produced. SurfacePrint asks how much that information can help distinguish human and language-model text.
The PDF includes all 13 main slides and 8 additional slides.
What the study shows
A small classifier learns from 97 measurements of raw text: punctuation, Markdown, math markup, code formatting and whitespace. Scoring a new passage requires no language-model calls.
On held-out texts from 17 model sources represented in training, SurfacePrint reaches a mean AUROC of 0.983, compared with 0.950 for Fast-DetectGPT. Both use the same test texts, but SurfacePrint learns from labeled examples; Fast-DetectGPT is a training-free detector.
A separately trained variant reaches 0.889 after selected whitespace cleanup. This comparison measures two trained variants; it does not test whether the original detector withstands edits.
How to read the results
AUROC measures how well scores rank generated text above human text: 0.5 is chance ranking and 1.0 is perfect. It is neither accuracy at a chosen cutoff nor a document’s probability of AI authorship.
Formatting offers an additional clue, with reliability that depends on the source and processing of the text. Transfer to unseen models is uneven, formatting is editable, and these experiments address non-adversarial detection. Practical use needs measured false-positive rates on representative documents; the score should not be sole evidence of authorship.
The manuscript is not yet publicly available.