#7 · Primary category: AI Usage & Cost Monitoring

agentops

agent agentops agents-sdk ai anthropic autogen cost-estimation crewai evals evaluation-metrics groq langchain llm mistral ollama openai openai-agents

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

Project last updated:06/25/26

GitHub Stars

5.8K

Forks

618

Contributors

49

License

MIT

Why we included this project

Teams shipping LLM-powered agents to production rarely have a clear view of what their agents actually did and what those calls cost. AgentOps closes that gap with a lightweight Python SDK that drops into most agent frameworks, including CrewAI, LangChain, AutoGen/AG2, and OpenAI Agents, and starts recording sessions after a couple of lines of setup. Once running, you get step-by-step execution traces for debugging and per-provider spend tracking across foundation models, plus the ability to replay full sessions when a run goes wrong. That makes it useful in two situations: during development, when you are trying to reproduce a bad agent run, and after launch, when a cost spike or a silent failure needs an explanation. The dashboard and API backend are also self-hostable, which matters for teams that cannot send traces to a cloud service. If you manage agents at any real scale and currently rely on scattered logs, this is a practical observability layer rather than a monitoring bolt-on.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category