{"id":"wildguard","title":"WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs","authors":[{"given":"Seungju","family":"Han"},{"given":"Kavel","family":"Rao"},{"given":"Allyson","family":"Ettinger"},{"given":"Liwei","family":"Jiang"},{"given":"Bill Yuchen","family":"Lin"},{"given":"Nathan","family":"Lambert"},{"given":"Yejin","family":"Choi"},{"given":"Nouha","family":"Dziri"}],"first_public":"2024-06-26","date_note":"First arXiv version.","date_source":"https://arxiv.org/abs/2406.18495","year":2024,"venue":"NeurIPS 2024 Datasets and Benchmarks","paper_url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/0f69b4b96a46f284b726fbd70f74fb3b-Abstract.html","arxiv":"2406.18495","category":"safety","relationship":"uses","dataset_version":"WildChat prompts form one component of a mixed-source moderation dataset; additional sources include LMSYS, annotator-written safety data, and synthetic adversarial examples.","summary":"Builds a moderation model that jointly identifies harmful requests, harmful responses, and refusals. WildChat contributes real user prompts and, through the WildTeaming pipeline, observed jailbreak tactics used to construct adversarial examples.","finding":"WildGuardMix contains roughly 92,000 examples, including a 5,299-item human-annotated test set. The resource supports both safety evaluation and moderation-model training.","evidence_url":"https://arxiv.org/html/2406.18495v3","evidence_note":"Section 3.1: WildChat sampling and tactic-based generation. Section 3.2: test annotation.","bibtex_key":"wildguard2024","record_url":"https://yuntiandeng.com/wildchat/papers/wildguard/","relationship_label":"Uses WildChat","citation_title":"WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs","citation_type":"paper-conference","bibtex_type":"inproceedings","citation_url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/0f69b4b96a46f284b726fbd70f74fb3b-Abstract.html","citation_year":2024,"citation_note":"NeurIPS 2024 proceedings paper.","doi":"10.52202/079017-0261","code_url":"https://github.com/allenai/wildguard","container_title":"Advances in Neural Information Processing Systems","volume":"37"}