Uses WildChat
WildCode Revisited: A Comprehensive Empirical Study on the Security of LLM-Generated Code
Extended arXiv preprint · First arXiv submission
Examines the security and quality of code generated in real user interactions, together with user intent and follow-up behavior. The extended study also tests security-oriented prompting strategies using prompts derived from the observed logs.
Data used
82,843 conversations containing generated code from an 837,989-conversation WildChat release; user-intent analyses use a filtered English subset of 34,478 conversations.
Findings
Security concerns are uncommon in the analyzed user requests, even when generated code has detected problems.
Section 3: extraction and language filtering. Sections 4–6: code, prompting, and user intent. Read the source.
Dates and versions
First arXiv version, originally titled WildCode: An Empirical Analysis of Code Generated by ChatGPT. ArXiv version history.
2025 arXiv preprint, extended June 2026 revision with updated title and author list.
Cite this paper
Kobra Khanmohammadi, Pooria Roy, Raphael Khoury, Abdelwahab Hamou-Lhadj, Wilfried Patrick Konan, Alexander Da Re, Nicholas Rebelo Melo. 2025. WildCode Revisited: A Comprehensive Empirical Study on the Security of LLM-Generated Code. arXiv preprint.
Show BibTeX
@misc{wildcode2025,
title = {{WildCode Revisited: A Comprehensive Empirical Study on the Security of LLM-Generated Code}},
author = {Khanmohammadi, Kobra and Roy, Pooria and Khoury, Raphael and Hamou-Lhadj, Abdelwahab and Konan, Wilfried Patrick and Da Re, Alexander and Melo, Nicholas Rebelo},
year = {2025},
doi = {10.48550/arXiv.2512.04259},
eprint = {2512.04259},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2512.04259}
}