private-gpt
Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.
PrivateGPT is an open-source API layer that transforms locally-running models into production-ready AI applications with advanced capabilities including RAG, MCP connectors, and built-in tools. It sits between your application and any OpenAI-compatible inference server (Ollama, llama.cpp, vLLM), providing higher-level primitives like document ingestion, retrieval with citations, web search, code execution, database querying, and structured data access. The project follows the Claude API specification and ships with a built-in workbench UI for testing, though the API itself is the core product. Originally a viral proof-of-concept for private document chat, it has evolved into a comprehensive backend for building local AI applications without cloud dependencies.
Key Features
Use Cases
- 01Building private AI assistants that process sensitive documents without sending data to cloud APIs
- 02Creating retrieval-augmented generation systems with local models and citations for enterprise knowledge bases
- 03Connecting Claude Desktop or Microsoft Office Claude add-ins to self-hosted models
- 04Developing automated workflows in n8n or other tools using local models as the backend
- 05Running agentic coding assistants in terminals with Claude Code or OpenCode using private infrastructure
- 06Querying proprietary databases and CSV files through natural language without cloud exposure
Related MCP Servers
View moremarkitdown
Python tool for converting files and office documents to Markdown.
firecrawl
The context API to search, scrape, and interact with the web at scale. 🔥
prompts.chat
f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.
langflow
Langflow is a powerful tool for building and deploying AI-powered agents and workflows.
private-gpt — FAQ
What is PrivateGPT?+
PrivateGPT is an open-source API layer that provides production-ready AI application features (RAG, tools, MCP connectors, database access) on top of locally-running language models. It connects to any OpenAI-compatible inference server and follows the Claude API specification.
How do I install PrivateGPT?+
Install via Homebrew on macOS (brew install private-gpt) or uv tool on Linux/Windows. You must have a running OpenAI-compatible LLM server (like Ollama) first, then configure PrivateGPT to connect via OPENAI_API_BASE environment variable.
Which AI clients work with PrivateGPT?+
PrivateGPT works natively with Claude Desktop, Claude Code, Claude for Microsoft 365, OpenCode, n8n, and any tool that supports OpenAI-compatible APIs. Official integration guides are available for Claude-based tools.
Do I need API keys to use PrivateGPT?+
No external API keys are required for core functionality since PrivateGPT runs on local models. You only need a local inference server like Ollama running. Optional features like web search may require their own API keys.
Is PrivateGPT free?+
Yes, PrivateGPT is open-source and free. The commercial Zylon platform builds on PrivateGPT with enterprise features like integrated infrastructure, governance, RBAC, and support.
Does PrivateGPT run models itself?+
No, PrivateGPT does not run models. It connects to external inference servers (Ollama, llama.cpp, vLLM, etc.) via the OpenAI-compatible API and adds application-layer features on top.
How do I install private-gpt?+
Open the source repository on GitHub and follow its README. private-gpt is a mcp server — MCP Agents Market links you directly to the official repo.
Is private-gpt free?+
private-gpt is an open-source project hosted on GitHub. Check the repository for its license and any usage requirements.