promptmap

Why we included this project

Before you put a custom LLM app in front of users, prompt injection is one of the failure modes worth testing for, and promptmap gives you a repeatable way to do that. It sends attack payloads to your target model or a live HTTP endpoint, then uses a separate judge model to decide whether any of them actually succeeded. You can run white-box tests by supplying your own system prompts, or black-box scans by pointing it at an endpoint with a small YAML config. It ships with more than fifty test rules covering prompt stealing, jailbreaking, harmful content generation, and bias, and lets you add your own pass/fail conditions. It works with OpenAI, Anthropic, Google, Grok, and local models via Ollama, so it fits into most stacks without forcing a provider switch. For teams that already have an evaluation or QA step, it's a straightforward pre-release check for catching prompt-related weaknesses early.