{"id":"wildchat-50m","title":"WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training","authors":[{"given":"Benjamin","family":"Feuer"},{"given":"Chinmay","family":"Hegde"}],"first_public":"2025-01-30","date_note":"First arXiv version.","date_source":"https://arxiv.org/abs/2501.18511","year":2025,"venue":"ICML 2025","paper_url":"https://proceedings.mlr.press/v267/feuer25a.html","arxiv":"2501.18511","code_url":"https://github.com/penfever/wildchat-50m","category":"training","relationship":"extends","dataset_version":"A third-party synthetic-response extension of WildChat-1M, retaining its user prompts and adding responses from open-weight models.","summary":"Uses WildChat prompts to compare responses from more than 50 open-weight model checkpoints and study the choice of synthetic-data generator. The authors also construct Re-Wild, a supervised fine-tuning mixture, and test it with Llama-3.1 8B.","finding":"The reported Re-Wild mixture outperforms the Tulu-3 SFT mixture on the paper's aggregate evaluations using 40% as many training samples.","evidence_url":"https://arxiv.org/html/2501.18511v1","evidence_note":"Section 2 identifies WildChat-1M as the prompt source and describes 54 checkpoints; Section 3 reports the SFT mixtures and evaluations.","bibtex_key":"feuer2025wildchat50m","record_url":"https://yuntiandeng.com/wildchat/papers/wildchat-50m/","relationship_label":"Extends WildChat","citation_title":"WildChat-50M: A Deep Dive Into the Role of Synthetic Data in Post-Training","citation_type":"paper-conference","bibtex_type":"inproceedings","container_title":"Proceedings of the 42nd International Conference on Machine Learning","citation_url":"https://proceedings.mlr.press/v267/feuer25a.html","citation_year":2025,"citation_note":"Cites the ICML 2025 proceedings paper in PMLR volume 267.","pages":"17100–17130","volume":"267","publisher":"PMLR"}