Uses WildChat
Navigating Rifts in Human-LLM Grounding: Study and Benchmark
ACL 2025 · First arXiv submission
Analyzes how users and assistants establish mutual understanding, alongside Bing Chat and human-human MultiWOZ conversations. It releases grounding annotations, a forecasting approach, and RIFTS to test when assistants should clarify, ask follow-up questions, or proceed.
Data used
Non-toxic English WildChat conversations, sampled at one conversation per user for the main grounding analysis; RIFTS selects initial prompts from WildChat.
Findings
RIFTS contains 1,740 tasks. The evaluated assistants rarely take useful grounding initiative; forecasting-based prompting improves their performance.
Section 4: data filters. Section 6 and Appendix A: benchmark construction and evaluation. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
ACL 2025 proceedings paper.
Cite this paper
Omar Shaikh, Hussein Mozannar, Gagan Bansal, Adam Fourney, Eric Horvitz. 2025. Navigating Rifts in Human-LLM Grounding: Study and Benchmark. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
Show BibTeX
@inproceedings{rifts2025,
title = {{Navigating Rifts in Human-LLM Grounding: Study and Benchmark}},
author = {Shaikh, Omar and Mozannar, Hussein and Bansal, Gagan and Fourney, Adam and Horvitz, Eric},
year = {2025},
booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
publisher = {Association for Computational Linguistics},
pages = {20832--20847},
doi = {10.18653/v1/2025.acl-long.1016},
url = {https://aclanthology.org/2025.acl-long.1016/}
}