Tracely-ai

Why we included this project

Most agent-observability tools stop at showing you what broke; Tracely acts on that result instead. It ingests agent traces over OTLP, scores each one with LLM-as-judge and structural checks, clusters repeated failures, and turns a bad run into a reproducible regression case that replays in CI with no API keys and no model spend. That case then blocks the pull request that would reintroduce the failure, and alerts land in Slack, email, or your own webhook. The conversation-grouped waterfall and per-agent metrics help teams running multi-agent setups see exactly which agent misfired and whether a fix held. The stack self-hosts with Postgres, ClickHouse, Redis, and MinIO, so teams that want evaluation and telemetry data to stay in their own infrastructure can keep it that way.