WildChat / research hub

Papers and connections

Research using WildChat

What can real conversations tell us about language models and the people who use them?

This selected bibliography includes the original dataset, exploration tools, and studies that use or extend WildChat. Each entry explains the relationship and identifies its source.

Literature checked 2026-10-19 · 8 papers · Download the bibliography

Showing 8 papers

2024Dataset

WildChat: 1M ChatGPT Interaction Logs in the Wild

Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, Yuntian Deng

Introduces consented, multilingual, multi-turn conversations with GPT-3.5 and GPT-4, together with timestamps and metadata for studying use across time and locations. The paper examines language and prompt diversity, toxicity and jailbreaks, and instruction tuning with WildLlama.

The paper reports 68 languages with more than 100 user prompts each and demonstrates instruction tuning with a Llama-2 7B model.

2024Evaluation

WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild

Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi

Turns challenging user requests from the WildChat collection into an evaluation set with task-specific checklists. The selection process combines language and quality filters, model difficulty estimates, and manual review, while reserving selected conversations from the public WildChat release.

For the top-ranked models evaluated in the paper, WB-Reward has a Pearson correlation of 0.98 with Chatbot Arena's human-voted Elo ratings.

2024User behavior

Developing Story: Case Studies of Generative AI's Use in Journalism

Natalie Grace Brigham, Chongjiu Gao, Tadayoshi Kohno, Franziska Roesner, Niloofar Mireshghallah

Matches conversations in WildChat to published articles from two news agencies, documenting how source material and generated drafts enter journalistic workflows. The study examines input materials, overlap with published text, and publication timing; its cases do not establish prevalence across journalism.

The authors match 78 conversations to 111 published articles and report a median output-to-publication ROUGE-L overlap of 0.62.

2024Privacy

Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wild

Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, Golnoosh Farnadi

Studies the tasks and contexts in which people disclose personal or sensitive information to chatbots. It combines automated PII detection with task and sensitive-topic taxonomies, releasing annotations that support analyses beyond conventional identifiers.

In the analyzed sample, sensitive disclosures extend beyond the categories recognized by PII detectors, including preferences, personal habits, and application-related information.

2024Exploration tools

WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wild

Yuntian Deng, Wenting Zhao, Jack Hessel, Xiang Ren, Claire Cardie, Yejin Choi

Provides composable search filters and an embedding map for inspecting WildChat and LMSYS-Chat-1M. Its case studies connect individual conversations to broader topic patterns and reproduce examples from research on journalism and personal disclosures.

The visual comparison identifies a Midjourney prompt-generation cluster in WildChat and a chemistry cluster in LMSYS-Chat-1M.

2025Training

WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training

Benjamin Feuer, Chinmay Hegde

Uses WildChat prompts to compare responses from more than 50 open-weight model checkpoints and study the choice of synthetic-data generator. The authors also construct Re-Wild, a supervised fine-tuning mixture, and test it with Llama-3.1 8B.

The reported Re-Wild mixture outperforms the Tulu-3 SFT mixture on the paper's aggregate evaluations using 40% as many training samples.

2025Information access

WildClaims: Conversational Information Access in the Wild(Chat)

Hideaki Joko, Shakiba Amirshahi, Charles L. A. Clarke, Faegheh Hasibi

Examines factual information conveyed by chatbot responses, including when the user's main task is editing or creative writing. It releases extracted claims and check-worthiness annotations to study information access beyond explicit questions.

WildClaims contains 121,905 extracted claims from 7,587 assistant utterances; estimated check-worthy-conversation prevalence varies substantially with the annotation method.

2026User behavior

Adopt ≠ Adapt: Longitudinal Analyses of LLM Conversations in the Wild

Rebecca M. M. Hicke, Kiran Tomlinson

Compares individual and population-level usage patterns in Bing Copilot with analyses of WildChat-4.8M. The study separates changes over a user's trajectory from differences between users, and uses the cross-platform comparison to examine how WildChat's sample differs.

WildChat users' intent and domain distributions more closely resemble high-activity Copilot users than low-activity users; hashed IPs remain an imperfect proxy for individuals.