2024Dataset
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, Yuntian Deng
Introduces consented, multilingual, multi-turn conversations with GPT-3.5 and GPT-4, together with timestamps and metadata for studying use across time and locations. The paper examines language and prompt diversity, toxicity and jailbreaks, and instruction tuning with WildLlama.
The paper reports 68 languages with more than 100 user prompts each and demonstrates instruction tuning with a Llama-2 7B model.
2024Evaluation
Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Faeze Brahman, Abhilasha Ravichander, Valentina Pyatkin, Nouha Dziri, Ronan Le Bras, Yejin Choi
Turns challenging user requests from the WildChat collection into an evaluation set with task-specific checklists. The selection process combines language and quality filters, model difficulty estimates, and manual review, while reserving selected conversations from the public WildChat release.
For the top-ranked models evaluated in the paper, WB-Reward has a Pearson correlation of 0.98 with Chatbot Arena's human-voted Elo ratings.
2024User behavior
Natalie Grace Brigham, Chongjiu Gao, Tadayoshi Kohno, Franziska Roesner, Niloofar Mireshghallah
Matches conversations in WildChat to published articles from two news agencies, documenting how source material and generated drafts enter journalistic workflows. The study examines input materials, overlap with published text, and publication timing; its cases do not establish prevalence across journalism.
The authors match 78 conversations to 111 published articles and report a median output-to-publication ROUGE-L overlap of 0.62.
2024Privacy
Niloofar Mireshghallah, Maria Antoniak, Yash More, Yejin Choi, Golnoosh Farnadi
Studies the tasks and contexts in which people disclose personal or sensitive information to chatbots. It combines automated PII detection with task and sensitive-topic taxonomies, releasing annotations that support analyses beyond conventional identifiers.
In the analyzed sample, sensitive disclosures extend beyond the categories recognized by PII detectors, including preferences, personal habits, and application-related information.
2024Exploration tools
Yuntian Deng, Wenting Zhao, Jack Hessel, Xiang Ren, Claire Cardie, Yejin Choi
Provides composable search filters and an embedding map for inspecting WildChat and LMSYS-Chat-1M. Its case studies connect individual conversations to broader topic patterns and reproduce examples from research on journalism and personal disclosures.
The visual comparison identifies a Midjourney prompt-generation cluster in WildChat and a chemistry cluster in LMSYS-Chat-1M.
2025Information access
Hideaki Joko, Shakiba Amirshahi, Charles L. A. Clarke, Faegheh Hasibi
Examines factual information conveyed by chatbot responses, including when the user's main task is editing or creative writing. It releases extracted claims and check-worthiness annotations to study information access beyond explicit questions.
WildClaims contains 121,905 extracted claims from 7,587 assistant utterances; estimated check-worthy-conversation prevalence varies substantially with the annotation method.