Extends WildChat
WildReward: Learning Reward Models from In-the-Wild Human Interactions
ACL 2026 · First arXiv submission
Trains reward models directly on implicit user feedback using ordinal regression. Its pipeline classifies feedback, filters ambiguous signals, and validates refusal cases before training; experiments examine calibration, cross-prompt score consistency, user diversity, and online DPO.
Data used
Filtered English and Chinese conversations from WildChat-4.8M, yielding approximately 186,000 feedback instances for the WildFB training data.
Findings
Broader user diversity improves reward modeling. WildReward models also improve online DPO results over the reward-model baseline tested in the paper.
Section 2: mining and training. Section 4: score consistency and user diversity. Appendix A: WildChat-4.8M language and quality filters. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
ACL 2026 proceedings paper.
Cite this paper
Hao Peng, Yunjia Qi, Xiaozhi Wang, Zijun Yao, Lei Hou, Juanzi Li. 2026. WildReward: Learning Reward Models from In-the-Wild Human Interactions. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
Show BibTeX
@inproceedings{wildreward2026,
title = {{WildReward: Learning Reward Models from In-the-Wild Human Interactions}},
author = {Peng, Hao and Qi, Yunjia and Wang, Xiaozhi and Yao, Zijun and Hou, Lei and Li, Juanzi},
year = {2026},
booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
publisher = {Association for Computational Linguistics},
pages = {18932--18948},
doi = {10.18653/v1/2026.acl-long.864},
url = {https://aclanthology.org/2026.acl-long.864/}
}