Uses WildChat
SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia
NAACL 2025 Findings · First arXiv submission
Builds Southeast Asian evaluation sets from local sources and uses real conversations to test how closely they reflect everyday requests. WildChat and LMSYS-Chat-1M supply the Wild Queries comparison corpus after language checks, length limits, and filtering.
Data used
Wild Queries combines filtered WildChat-1M and LMSYS-Chat-1M inputs in Indonesian, Thai, and Vietnamese; the paper reports 4,658 queries across the two sources.
Findings
In embedding-based comparisons, the locally constructed SeaExam and SeaBench are closer to the observed query distribution than translated MMLU and MT-Bench.
Section 3.1: Wild Queries construction and distribution comparisons. Appendix B and Table 10: language breakdown. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
NAACL 2025 Findings proceedings paper.
Cite this paper
Chaoqun Liu, Wenxuan Zhang, Jiahao Ying, Mahani Aljunied, Anh Tuan Luu, Lidong Bing. 2025. SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia. Findings of the Association for Computational Linguistics: NAACL 2025.
Show BibTeX
@inproceedings{seaexamseabench2025,
title = {{SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia}},
author = {Liu, Chaoqun and Zhang, Wenxuan and Ying, Jiahao and Aljunied, Mahani and Luu, Anh Tuan and Bing, Lidong},
year = {2025},
booktitle = {Findings of the Association for Computational Linguistics: NAACL 2025},
publisher = {Association for Computational Linguistics},
pages = {6134--6151},
doi = {10.18653/v1/2025.findings-naacl.341},
url = {https://aclanthology.org/2025.findings-naacl.341/}
}