WildChat / research hub

Uses WildChat

Dataset Signatures in Human-LLM Interactions and User Modeling

Joseph Suh, Serina Chang

arXiv preprint · First arXiv submission

Studies how conversation sources differ and how those differences propagate into trained user simulators and assistant evaluations. Uses the AI Observatory taxonomy to compare dataset categories alongside learned source classifiers.

Data used

Seven conversation datasets, including WildChat-1M and WildChat-4.8M, with controlled comparisons to LMSYS-Chat-1M, ShareGPT, and other interaction sources.

Findings

Classifiers identify the source from user messages above chance even after taxonomy matching. Dataset choice changes user-model behavior and downstream evaluations.

Sections 2–3 and Appendices A–C: dataset sampling, source classifiers, taxonomy matching, and user modeling. Cites WildChat, WildBench, and AI Observatory. Read the source.

Dates and versions

First arXiv version. ArXiv version history.

arXiv preprint.

Cite this paper

Joseph Suh, Serina Chang. 2026. Dataset Signatures in Human-LLM Interactions and User Modeling. arXiv preprint.

Show BibTeX
@misc{datasetsignatures2026,
  title = {{Dataset Signatures in Human-LLM Interactions and User Modeling}},
  author = {Suh, Joseph and Chang, Serina},
  year = {2026},
  doi = {10.48550/arXiv.2610.05534},
  eprint = {2610.05534},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.05534}
}

Reuse this entry