Extends WildChat
WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training
ICML 2025 · First arXiv submission
Uses WildChat prompts to compare responses from more than 50 open-weight model checkpoints and study the choice of synthetic-data generator. The authors also construct Re-Wild, a supervised fine-tuning mixture, and test it with Llama-3.1 8B.
Dataset and scope
A third-party synthetic-response extension of WildChat-1M, retaining its user prompts and adding responses from open-weight models.
Selected finding
The reported Re-Wild mixture outperforms the Tulu-3 SFT mixture on the paper's aggregate evaluations using 40% as many training samples.
Section 2 identifies WildChat-1M as the prompt source and describes 54 checkpoints; Section 3 reports the SFT mixtures and evaluations. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
Cites the ICML 2025 proceedings paper in PMLR volume 267.
The annotation describes the linked source version. Consult the paper for its full methods, assumptions, and limitations.
Cite this paper
Benjamin Feuer, Chinmay Hegde. 2025. WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training. Proceedings of the 42nd International Conference on Machine Learning.
Show BibTeX
@inproceedings{feuer2025wildchat50m,
title = {{WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training}},
author = {Feuer, Benjamin and Hegde, Chinmay},
year = {2025},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
publisher = {PMLR},
volume = {267},
pages = {17100--17130},
url = {https://proceedings.mlr.press/v267/feuer25a.html}
}Reuse this entry
JSON recordFull reading list in MarkdownFull bibliography
Annotations are a reading aid prepared for this hub. The linked authors’ work is the source for research claims. About the selection and corrections.