Uses WildVis
Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
ICML 2025 · First arXiv submission
Uses conversations found through WildVis to study safety failures during multi-step and multilingual interactions. The paper introduces HarmScore to assess the actionability and informativeness of harmful responses, and tests these interaction patterns across four safety benchmarks.
Data used
Real-world conversations retrieved through WildVis motivate the study. Experiments use HarmBench, AdvBench, SORRY-Bench, and MedSafetyBench.
Findings
Figure 2 uses WildVis conversations to illustrate the interaction patterns tested in the safety evaluation.
Section 4 and Figure 2: WildVis conversations. Section 5: benchmark evaluation. Read the source.
Dates and versions
First arXiv version. ArXiv version history.
ICML 2025 proceedings paper, PMLR volume 267.
Cite this paper
Yik Siu Chan, Narutatsu Ri, Yuxin Xiao, Marzyeh Ghassemi. 2025. Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions. Proceedings of the 42nd International Conference on Machine Learning.
Show BibTeX
@inproceedings{chan2025speakeasy,
title = {{Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions}},
author = {Chan, Yik Siu and Ri, Narutatsu and Xiao, Yuxin and Ghassemi, Marzyeh},
year = {2025},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
publisher = {PMLR},
volume = {267},
pages = {7306--7331},
url = {https://proceedings.mlr.press/v267/chan25b.html}
}