deepeval

Why we included this project

DeepEval treats LLM evaluation the way a test runner treats code: you write assertions against your app's outputs and get scored results back. The metric library covers hallucination, answer relevancy, faithfulness, agent task completion, tool correctness, and multi-turn conversation quality, and many of those metrics run locally, so you are not locked into paying a judge model for every check. That makes it a good fit for teams iterating on RAG pipelines, agents, or chatbots who want to compare model versions, prompts, and architectures before committing to a change. It also includes metrics for multimodal outputs and MCP-based agents, so it is not limited to plain text chat. If you want evaluation to be a repeatable part of your development loop instead of an afterthought, this is a practical place to start.