giskard-oss

Why we included this project

For teams shipping LLM agents, RAG pipelines, or a single model endpoint, this is a practical way to check behavior before it reaches users. It pairs a checks package for writing evals with a scan layer that auto-generates adversarial test suites, covering prompt injection, jailbreaks, harmful content, and other OWASP LLM Top-10 threats. The eval side ranges from simple assertions up to LLM-as-judge groundedness and conformity checks that cope with non-deterministic output, while the scan helps you red-team known failure modes without hand-writing every attack. The v3 rewrite is modular and async-first, so you pull in only the pieces you need and can wrap any callable as the target under test. If you want evaluation and security testing in one pip-installable package instead of juggling separate tools, this is a solid, actively maintained place to start.