opik

Why we included this project

Opik is a self-hostable observability and evaluation platform for LLM applications, built by Comet. It records every trace, from a single model call up to the full tree of tool calls in a multi-step agent, which is what you need when a response goes wrong and you have to figure out why. On the evaluation side it lets you build datasets, run experiments, and apply LLM-as-a-judge metrics for hallucination detection and RAG answer relevance, and you can wire those checks into CI with a PyTest integration. Since the whole platform, backend included, is Apache-2.0 and free to self-host, teams can keep their trace data inside their own infrastructure. If you are taking an agent or RAG system from prototype to production and want debugging depth and repeatable evaluation in one tool, it is worth a look.