langwatch
The platform for LLM evaluations and AI agent testing
The LangWatch MCP server brings comprehensive LLM evaluation and AI agent testing capabilities to Model Context Protocol clients. It provides a unified platform for running agent simulations, evaluating performance, tracing agent behavior, and monitoring production deployments. Teams can test agents against realistic scenarios, measure quality metrics, and optimize prompts without switching tools. The server integrates with major AI frameworks through OpenTelemetry standards and includes an AI gateway for governance and cost control.
Key Features
Use Cases
- 01Running regression tests on AI agents before production deployment to identify breaking points
- 02Monitoring LLM-powered applications in production with detailed trace analysis and performance metrics
- 03Simulating realistic user scenarios to evaluate agent behavior across different edge cases
- 04Optimizing prompt engineering and model selection through integrated evaluation workflows
- 05Managing AI infrastructure costs with hierarchical budgets and provider fallback routing
- 06Collaborating across teams with domain experts annotating failures and reviewing agent decisions
Related MCP Servers
View moremarkitdown
Python tool for converting files and office documents to Markdown.
firecrawl
The context API to search, scrape, and interact with the web at scale. 🔥
prompts.chat
f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.
langflow
Langflow is a powerful tool for building and deploying AI-powered agents and workflows.
langwatch — FAQ
What is the LangWatch MCP server?+
The LangWatch MCP server is a Model Context Protocol integration that provides LLM evaluation, agent testing, and observability capabilities to MCP-compatible clients like Claude Desktop. It enables developers to trace, evaluate, and optimize AI agents directly from their development environment.
How do I install the LangWatch MCP server?+
For local development, run 'npx @langwatch/server' which automatically installs dependencies and starts services. For production use, you can deploy via Docker Compose, Kubernetes Helm, or cloud-specific configurations. The MCP server integration requires adding the server configuration to your MCP client settings.
Which AI clients work with the LangWatch MCP server?+
The LangWatch MCP server works with Claude Desktop and other MCP-compatible clients. The platform also integrates with major AI frameworks including LangChain, LangGraph, Vercel AI SDK, CrewAI, and platforms like LangFlow, Flowise, and n8n.
Do I need API keys to use LangWatch?+
For the cloud version, you'll need to create a free LangWatch account and obtain an API key from your project settings. Self-hosted deployments generate local secrets automatically. Integrating with LLM providers requires their respective API keys.
Is LangWatch free to use?+
LangWatch offers a free cloud tier and the core platform is Apache 2.0 licensed for self-hosting. The SDKs (TypeScript, Python, MCP server) are MIT licensed. Enterprise features like SSO, SCIM provisioning, and audit logs require a commercial license in production.
What are the prerequisites for running LangWatch locally?+
The quickest setup requires only Node.js installed on your machine. The CLI command 'npx @langwatch/server' automatically installs PostgreSQL, Redis, ClickHouse, and other dependencies into the '~/.langwatch/' directory. For Docker deployment, Docker Compose is required.
How do I install langwatch?+
Open the source repository on GitHub and follow its README. langwatch is a mcp server — MCP Agents Market links you directly to the official repo.
Is langwatch free?+
langwatch is an open-source project hosted on GitHub. Check the repository for its license and any usage requirements.