jailbreak_llms

Why we included this project

Security teams working on prompt hardening will get real mileage out of this dataset. It backs a peer-reviewed CCS 2024 paper and collects 15,140 actual ChatGPT prompts pulled from Reddit, Discord, prompt-sharing sites, and open-source repos, with about 1,405 flagged as jailbreak attempts. The repo also includes a 390-question set of forbidden scenarios derived from OpenAI's usage policy and a scoring script to see how well a model resists those requests. You can pull the prompts straight into your pipeline via the Hugging Face Datasets library, which makes it easy to benchmark defenses or build red-team evals. Because these are real-world attacks rather than synthetic examples, they give you a grounded baseline for testing whether your guardrails survive actual use.