Uses WildVis
Trust Me, I’m Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer
EMNLP 2025 Findings · First arXiv submission
Uses WildVis to check that prompt variations in a hallucination study resemble real human requests. The paper studies cases where models give confident incorrect answers after small prompt changes despite consistently answering the original question correctly.
Data used
Queries WildChat and LMSYS-Chat-1M through WildVis to validate help-seeking prompt styles; the main experiments use Natural Questions and TriviaQA.
Findings
The authors use WildVis searches and WildChat examples to validate the help-seeking language used in their evaluation prompts.
Section 3.1: prompt construction and validation. Appendix F.2: WildVis searches of WildChat and LMSYS-Chat-1M. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
EMNLP 2025 Findings proceedings paper.
Cite this paper
Adi Simhi, Itay Itzhak, Fazl Barez, Gabriel Stanovsky, Yonatan Belinkov. 2025. Trust Me, I’m Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer. Findings of the Association for Computational Linguistics: EMNLP 2025.
Show BibTeX
@inproceedings{trustmeimwrong2025,
title = {{Trust Me, I’m Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answer}},
author = {Simhi, Adi and Itzhak, Itay and Barez, Fazl and Stanovsky, Gabriel and Belinkov, Yonatan},
year = {2025},
booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2025},
publisher = {Association for Computational Linguistics},
pages = {14665--14688},
doi = {10.18653/v1/2025.findings-emnlp.792},
url = {https://aclanthology.org/2025.findings-emnlp.792/}
}