Uses WildChat
RewardBench 2: Advancing Reward Model Evaluation
ICLR 2026 · First arXiv submission
Builds a reward-model benchmark around previously unseen human prompts from WildChat, combining quality and topic filtering with manual verification. It evaluates reward models across factuality, precise instruction following, math, safety, focus, and ties, and studies how benchmark scores relate to best-of-N sampling and RLHF.
Data used
Previously unreleased WildChat prompts supply approximately 70% of the 1,865-prompt benchmark.
Findings
WildChat supplies the human-prompt domains: factuality (475), precise instruction following (160), math (183), and focus (495). The safety and ties domains use other prompt sources.
Section 3 and Table 2: WildChat prompts and benchmark composition. Sections 4–5: reward models and downstream use. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
2025 arXiv preprint; published at ICLR 2026.
Cite this paper
Saumya Malik, Valentina Pyatkin, Sander Land, Jacob Morrison, Noah A. Smith, Hannaneh Hajishirzi, Nathan Lambert. 2025. RewardBench 2: Advancing Reward Model Evaluation. arXiv preprint.
Show BibTeX
@misc{malik2025rewardbench2,
title = {{RewardBench 2: Advancing Reward Model Evaluation}},
author = {Malik, Saumya and Pyatkin, Valentina and Land, Sander and Morrison, Jacob and Smith, Noah A. and Hajishirzi, Hannaneh and Lambert, Nathan},
year = {2025},
doi = {10.48550/arXiv.2506.01937},
eprint = {2506.01937},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2506.01937}
}