Uses WildChat
Dataset Signatures in Human-LLM Interactions and User Modeling
arXiv preprint · First arXiv submission
Studies how conversation sources differ and how those differences propagate into trained user simulators and assistant evaluations. Uses the AI Observatory taxonomy to compare dataset categories alongside learned source classifiers.
Data used
Seven conversation datasets, including WildChat-1M and WildChat-4.8M, with controlled comparisons to LMSYS-Chat-1M, ShareGPT, and other interaction sources.
Findings
Classifiers identify the source from user messages above chance even after taxonomy matching. Dataset choice changes user-model behavior and downstream evaluations.
Sections 2–3 and Appendices A–C: dataset sampling, source classifiers, taxonomy matching, and user modeling. Cites WildChat, WildBench, and AI Observatory. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
arXiv preprint.
Cite this paper
Joseph Suh, Serina Chang. 2026. Dataset Signatures in Human-LLM Interactions and User Modeling. arXiv preprint.
Show BibTeX
@misc{datasetsignatures2026,
title = {{Dataset Signatures in Human-LLM Interactions and User Modeling}},
author = {Suh, Joseph and Chang, Serina},
year = {2026},
doi = {10.48550/arXiv.2610.05534},
eprint = {2610.05534},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05534}
}