agentops
Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI
AgentOps is a Python SDK for comprehensive AI agent observability, monitoring, and debugging that tracks LLM costs, session replays, and agent performance. It provides developers with analytics tools to monitor AI agents from prototype to production, offering session-wide metrics, real-time debugging, and cost management. The SDK integrates natively with major frameworks including CrewAI, AG2 (AutoGen), LangChain, OpenAI Agents SDK, Camel AI, and others through simple decorators or automatic instrumentation. AgentOps enables tracking of agent operations, tool usage, API calls, and multi-agent interactions with minimal code changes.
Key Features
Use Cases
- 01Monitor and debug multi-agent systems built with CrewAI or AG2 frameworks
- 02Track LLM API costs and usage across OpenAI, Anthropic, Cohere, and Mistral providers
- 03Analyze agent performance bottlenecks with event latency analysis and execution pricing
- 04Replay agent sessions to understand decision-making workflows and failure points
- 05Evaluate agent reliability and success rates across different environments
- 06Implement observability for production AI agents with minimal code instrumentation
Related Skills
View moresuperpowers
An agentic skills framework & software development methodology that works.
skills
Skills for Real Engineers. Straight from my .agents directory.
skills
Public repository for Agent Skills
ponytail
Makes your AI agent think like the laziest senior dev in the room. The best code is the code you never wrote.
agentops — FAQ
What is AgentOps?+
AgentOps is a Python SDK for AI agent observability that provides monitoring, debugging, and analytics for agents built with frameworks like CrewAI, LangChain, AG2, and OpenAI Agents SDK. It tracks LLM costs, session replays, and performance metrics with minimal code integration.
How do I install AgentOps?+
Install via pip with 'pip install agentops', then add two lines to your code: import agentops and agentops.init('YOUR_API_KEY') at the beginning, and agentops.end_session('Success') at the end. For framework-specific installations like CrewAI, use 'pip install crewai[agentops]'.
Do I need an API key to use AgentOps?+
Yes, you need a free AgentOps API key which you can obtain from the AgentOps dashboard at app.agentops.ai/settings/projects. Set it as an environment variable (AGENTOPS_API_KEY) or pass it directly to agentops.init().
Which AI frameworks does AgentOps support?+
AgentOps natively integrates with CrewAI, AG2 (AutoGen), OpenAI Agents SDK (Python & TypeScript), LangChain, Camel AI, LlamaIndex, Cohere, Anthropic, Mistral, LiteLLM, Llama Stack, and SwarmZero. It supports both synchronous and asynchronous operations.
Is AgentOps free and open source?+
The AgentOps application is open source under the MIT license, and the SDK can be used with a free API key. The platform offers both cloud-hosted and self-hosting options for running the full dashboard and API backend.
Can I use AgentOps with Claude or ChatGPT?+
AgentOps tracks agents that use Claude (via Anthropic SDK integration) or ChatGPT (via OpenAI integration), but it's a monitoring SDK, not a client. It works with agents built using these LLMs within supported frameworks like LangChain or custom agent implementations.
How do I install agentops?+
Open the source repository on GitHub and follow its README. agentops is a skill — MCP Agents Market links you directly to the official repo.
Is agentops free?+
agentops is an open-source project hosted on GitHub. Check the repository for its license and any usage requirements.