PrivateGPT

PrivateGPT is a free, open-source API layer for developers building private AI applications on their own infrastructure. It does not run models itself; instead, it connects to a separate OpenAI-compatible inference server, which may be local or remote. The Messages API supports streaming, asynchronous processing, and token counting. Document workflows include file and artifact ingestion, retrieval with citations, and access to databases and tabular data such as CSVs. The documented tool set also includes web search and extraction, code execution, MCP connectors, and custom tools. A browser workbench at `/ui` lets developers try demos and inspect requests, but it stores chats and settings in browser localStorage and has no access control; the documentation calls it a local demonstrator. Installation options include a Python package, source, and Docker, with the application running as a local server. Package installation requires Python 3.11; Python 3.10 and 3.12 or later are unsupported. The maker documents connections to Ollama, LM Studio, llama.cpp, and vLLM, and links to a community Discord. The maker identifies itself as Zylon.

PrivateGPT supplies developer-facing APIs and document tools while leaving model inference to a separate compatible server. Its local demonstrator has no access control, and package users must meet the Python 3.11 requirement.

It is intended for developers seeking an open-source application layer and API for private AI products. Users need a separate compatible inference server and, for package installation, Python 3.11.

PrivateGPT is a free, open-source application layer for developers assembling private AI products around models they choose to operate. It is strongest when document retrieval, structured data, and tool calls need one API, but it is a poor fit for anyone expecting a complete hosted model service.

Overview

PrivateGPT runs as a local server and can be installed as a Python package, from source, or with Docker. It connects to OpenAI-compatible inference servers such as Ollama, LM Studio, llama.cpp, and vLLM instead of executing models itself. That modular design lets a team change inference providers without replacing its application layer, while adding the operational work of deploying and maintaining a separate server.

The project is open source and free. Application data—including the vector store, ingested documents, models, and caches—is stored under a local home directory by default. Remote compatible servers are also supported, so local storage does not automatically mean inference remains on the same machine. Optional observability can emit traces for language-model calls, embedding requests, and retrieval steps to the console, Arize Phoenix, or Opik.

Key features

The Messages API handles streaming, asynchronous processing, and token counting. Those controls suit applications that need responsive output, background work, or usage accounting rather than a single synchronous prompt. File and artifact ingestion feed retrieval-augmented generation, while citations give users a way to follow answers back to source material.

PrivateGPT also exposes structured access to databases and tabular data such as CSV files. Documented tools include database querying, CSV analysis, web search and extraction, code execution, MCP connectors, and custom tools. MCP usage is documented through the Messages API, and integrations are listed for Claude Code, OpenCode, VS Code, Cline, and n8n. The breadth is valuable for a developer building an application workflow, although the compatibility table marks some skills as basic or still in progress.

A browser workbench at /ui supports demonstrations, request inspection, and API experiments. It is explicitly a local demonstrator: chats and settings live in browser localStorage, and there is no access control. Do not treat it as a multi-user production console or expose it without adding your own security layer.

Pricing

There are no paid tiers, seats, usage quotas, or hosted-service terms in this plan. The trade-off is infrastructure: model serving, storage, and operations remain your responsibility, and PrivateGPT itself does not supply inference.

Platforms

PrivateGPT is offered for API, Linux, macOS, self-hosted deployments, web, and Windows. The package installation guide requires Python 3.11; Python 3.10 and Python 3.12 or later are unsupported. Docker is documented as another installation route, which can be preferable when a project wants a repeatable server deployment.

Who it's for

Choose PrivateGPT if you are building a private AI application and want an open-source API spanning retrieval, citations, data analysis, and tools while retaining control of the model server and storage. It is also a sensible foundation for teams already using an OpenAI-compatible provider and needing a document-focused application layer.

Look elsewhere if you need a hosted model, a secured collaborative workbench, or a turnkey product with no separate inference component. The unsupported OAuth or organizations capabilities and the workbench's absent access control make it unsuitable as-is for many shared production environments.

Pros and cons

Pros

• Broad developer API: Streaming, asynchronous jobs, and token counting support production application patterns.
• Strong retrieval workflow: File and artifact ingestion, retrieval-augmented generation, and citations connect answers to source documents.
• Flexible infrastructure: Ollama, LM Studio, llama.cpp, vLLM, and other OpenAI-compatible servers can provide inference.
• Useful tool surface: Database and CSV access, web extraction, code execution, MCP, and custom tools cover varied workflows.
• No license charge: The open-source, self-hosted plan costs nothing.

Cons

• Inference is separate: You must deploy and maintain a compatible model server; PrivateGPT is not a model host.
• Workbench security is limited: Browser localStorage and no access control restrict it to local demonstrations and testing.
• Strict Python support: Package installation requires Python 3.11 and rejects 3.10 and 3.12 or later.
• Incomplete platform controls: OAuth and organizations are unsupported, while some skills remain basic or in progress.

Alternatives

Browse the Retrieval-Augmented Generation Tools list for more document-focused options.

AnythingLLM is the better choice when Android support or a freemium hosted path matters alongside self-hosting; its Basic plan is $50 a month billed monthly.

Graphiti suits teams wanting a focused, Apache-2.0, self-managed framework rather than this broader application layer.

OpenRAG fits developers seeking an AGPL-3.0 self-hosted RAG deployment with no feature held back.

Quivr is a paid API and web product with free named plans for knowledge management and support-data insights.

txtai is a lean Apache-2.0 option installable with pip or Docker.

Dify is preferable when a managed-style sandbox with explicit workspace, app, document, storage, and request caps is more useful than assembling your own inference stack.

Haystack offers an open-source Python framework, while Haystack Enterprise Platform adds a web Studio with cloud deployment and defined workspace limits.

Verdict

PrivateGPT is a strong pick for developers who want an open-source, self-hosted API tying local or selected inference servers to retrieval, citations, data, and tools. Choose it for architectural control and a wide integration surface; choose another product if you need hosted models, built-in access control, or a turnkey collaborative service.