{"id":"wildteaming","title":"WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models","authors":[{"given":"Liwei","family":"Jiang"},{"given":"Kavel","family":"Rao"},{"given":"Seungju","family":"Han"},{"given":"Allyson","family":"Ettinger"},{"given":"Faeze","family":"Brahman"},{"given":"Sachin","family":"Kumar"},{"given":"Niloofar","family":"Mireshghallah"},{"given":"Ximing","family":"Lu"},{"given":"Maarten","family":"Sap"},{"given":"Yejin","family":"Choi"},{"given":"Nouha","family":"Dziri"}],"first_public":"2024-06-26","date_note":"First arXiv version.","date_source":"https://arxiv.org/abs/2406.18510","year":2024,"venue":"NeurIPS 2024","paper_url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/54024fca0cef9911be36319e622cde38-Abstract-Conference.html","arxiv":"2406.18510","category":"safety","relationship":"extends","dataset_version":"Single-turn WildChat and LMSYS-Chat-1M examples filtered for potentially harmful interactions; WildChat contributes 8,981 adversarial queries to the tactic inventory.","summary":"Mines naturally occurring jailbreak tactics and combines them into adversarial tests and synthetic safety-training examples. The resulting WildJailbreak resource contains harmful and benign requests in both direct and adversarial forms, enabling experiments on safety and over-refusal.","finding":"WildChat and LMSYS-Chat-1M yield 5,688 tactic clusters. The synthetic WildJailbreak dataset contains approximately 262,000 prompt-response pairs.","evidence_url":"https://arxiv.org/html/2406.18510v1","evidence_note":"Section 2 and Table 1: WildChat and LMSYS contributions. Sections 4–5: WildJailbreak and safety-training experiments.","bibtex_key":"wildteaming2024","record_url":"https://yuntiandeng.com/wildchat/papers/wildteaming/","relationship_label":"Extends WildChat","citation_title":"WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models","citation_type":"paper-conference","bibtex_type":"inproceedings","citation_url":"https://proceedings.neurips.cc/paper_files/paper/2024/hash/54024fca0cef9911be36319e622cde38-Abstract-Conference.html","citation_year":2024,"citation_note":"NeurIPS 2024 proceedings paper.","doi":"10.52202/079017-1493","code_url":"https://github.com/allenai/wildteaming","container_title":"Advances in Neural Information Processing Systems","volume":"37"}