Uses WildChat
From Chat Logs to Collective Insights: Aggregative Question Answering
EMNLP 2025 · First arXiv submission
Introduces aggregative question answering: answering questions that require combining evidence across many conversations. WildChat-AQA pairs conversation text and metadata with inferred topics and keywords, and compares retrieval, fine-tuning, and the PROBE retrieval method.
Data used
A filtered WildChat-Full release of roughly 990,000 conversations, reduced to 182,330 conversations after deduplication, length limits, and an active-user requirement.
Findings
The benchmark contains 6,027 questions over 182,330 conversations. PROBE improves over standard retrieval, with retrieval and reasoning errors remaining in the evaluated systems.
Section 2 and Appendix E: construction, model-generated questions, and human quality checks. Section 4: ranking quality, retrieval, and computational cost. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
EMNLP 2025 proceedings paper.
Cite this paper
Wentao Zhang, Woojeong Kim, Yuntian Deng. 2025. From Chat Logs to Collective Insights: Aggregative Question Answering. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.
Show BibTeX
@inproceedings{wildchataqa2025,
title = {{From Chat Logs to Collective Insights: Aggregative Question Answering}},
author = {Zhang, Wentao and Kim, Woojeong and Deng, Yuntian},
year = {2025},
booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
publisher = {Association for Computational Linguistics},
pages = {32811--32850},
doi = {10.18653/v1/2025.emnlp-main.1667},
url = {https://aclanthology.org/2025.emnlp-main.1667/}
}