ChainForge

ChainForge is a free, open-source visual environment for prompt engineering, language model evaluation and experimentation. It can query multiple models and compare their responses across prompt variations, model choices and settings. Users can score responses with code-based or LLM-based evaluators and plot results. Chat Turn nodes support continuing several conversations, templating messages and passing a conversation from one model to another. ChainForge imports JSONL, XLSX and CSV spreadsheets and exports inspected response data as XLSX. Listed providers include OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama; local installations can add custom providers written as Python completion functions. It is available on the web or self-hosted. The web version has limited features and requires users to enter API keys each time they load it; the documentation says keys are not stored. Local use supports environment-based API keys, Python evaluator code and locally hosted Alpaca or Llama models via Dalai. ChainForge is an open beta in active development. Its listed browser support is Chrome, Firefox, Edge and Brave.

ChainForge provides a flexible workspace for comparing model responses and evaluating prompt experiments, with additional capabilities in local installations. Its beta status, web feature limits and unrecoverable encryption password are useful constraints to consider.

ChainForge suits people comparing prompts and language model responses, including those who want to score and plot results. Self-hosting is relevant for users who need the local features or custom Python providers.

ChainForge is an open-source visual workspace for designing prompt experiments and evaluating language-model responses. It suits developers, researchers and advanced prompt practitioners who want to compare several models systematically, especially when they can run the local version. The main trade-off is clear: the free software is unusually flexible, but browser use is narrower and local encryption has no password recovery.

Overview

ChainForge treats prompt work as an experiment you can repeat and inspect rather than a collection of isolated chat sessions. A visual programming approach lets you vary prompts, models and settings, then review outputs together. That is a good fit for teams investigating response quality across providers or building an evaluation workflow around their own datasets. It is less suitable for someone seeking automated prompt optimization or a polished, hosted prompt-management service; optimization is manual and the project is still an open beta in active development.

Key features

The core workflow can query multiple LLMs at once and compare responses across prompt variations, models and model settings. This makes changes easier to judge across a group of outputs instead of trusting a single example. Responses can be scored with code-based or LLM-based evaluators, and results can be plotted, giving technically minded users a way to turn qualitative comparisons into a more structured review.

Chat Turn nodes extend testing beyond one-shot prompts. They can continue multiple conversations, template chat messages and pass a conversation from one LLM to another, which is useful for checking multi-turn behavior and provider handoffs. ChainForge also imports JSONL, XLSX and CSV files and exports inspected response data as XLSX, so existing test sets can feed the same workflow and results can move into spreadsheet analysis.

Provider coverage includes OpenAI, Anthropic, Google Gemini and PaLM, Hugging Face, Azure OpenAI, Amazon Bedrock and Ollama. A local installation can add custom providers as Python completion functions through its Custom Providers settings. That extensibility is a strong reason to install locally, but it assumes comfort with Python and local model configuration rather than a purely no-code setup.

The web edition can create a unique share link for an experiment, useful when showing work to colleagues. It has a limited feature set, however. The full local edition adds API keys from environment variables, Python evaluator nodes and locally hosted Alpaca or Llama models through Dalai, with Ollama support also available locally. The web version does not retain keys in cookies, local storage or the server, so keys must be entered each time it is loaded.

Pricing

There is no paid tier or seat schedule to weigh against the free release. You still pay any underlying model or API charges, and the practical cost depends on which providers and local models you use. The free plan is therefore the natural choice for individual experimentation, research and teams willing to operate their own installation. Readers who need hosted administration, formal usage quotas or vendor support should look at a commercial alternative instead.

Platforms

ChainForge is offered as self-hosted software and a web application. The installation guide supports Chrome, Firefox, Edge and Brave; the beta does not support other browsers. Local mode is the more capable option, while the browser mode favors quick access and shareable experiments at the expense of features.

Local security controls can encrypt settings or all locally saved files through secure-mode options. The password is stored only in hashed form and cannot be recovered, so forgetting it means losing access to encrypted flows or settings. GitHub issues and a Google Form are the stated channels for questions, comments and feature requests.

Who it's for

Choose ChainForge if you need a free, inspectable environment for comparing prompts and models, scoring outputs, testing conversations and plugging in custom providers. It is particularly well suited to technical users who can run Python-based evaluators or local models and who value control over where experiments and credentials live.

It is not the best match for a nontechnical user who wants automated optimization, a fully hosted workspace or guaranteed browser coverage. The open-beta status, repeated key entry in the web app and unrecoverable local encryption password all add operational responsibility.

Pros and cons

• Pros: free open-source distribution avoids subscription and seat costs.
• Pros: side-by-side multi-model and multi-prompt queries make comparative testing practical.
• Pros: code and LLM scorers, plots and spreadsheet import/export support repeatable evaluation work.
• Pros: local custom providers, environment-loaded keys and Ollama or other local-model workflows extend it beyond built-in integrations.
• Cons: the web version omits capabilities found in the local release and requires API keys to be set again on each load.
• Cons: manual optimization means users must design and interpret experiments themselves.
• Cons: a forgotten encryption password permanently blocks encrypted flows or settings.
• Cons: open-beta status and limited browser support make it a less predictable choice for broad organizational rollout.

Alternatives

PromptHelm is the better choice when you want a freemium gateway with Android, iOS, API, web and self-hosted access, but its free tier is capped at 10,000 gateway calls per month, 10 prompts, 20 versions per prompt, one seat and seven-day log retention.

Pick Vellum instead for a broader hosted-and-desktop footprint and a free entry that includes initial tasks with pay-as-you-go credits; its Mighty plan costs $30 a month.

Agenta suits teams wanting a hosted or self-hosted freemium service, with a forever-free Hobby plan for two team members, 5,000 agent runs and 20 evaluations per month.

Choose Maxim AI for a developer-oriented hosted workflow with up to three seats, 10,000 logs per month and three-day retention on its free plan.

Prompt Manage is preferable for straightforward prompt organization and public sharing, with 25 prompts and 10 runs per month free.

Use PromptLayer when you need a managed API-centric workspace with five users, 2,500 requests and 250 evaluation-cell executions per month on its free plan.

Apastra is another free, self-hosted, Apache-2.0 local-first framework.

Pick Braintrust for a hosted evaluation service whose Starter plan includes 1 GB of processed data, 10,000 scores, 14-day retention and unlimited users, projects and datasets.

For broader discovery, browse AI Prompt Management Software and AI Prompt Generators.

Verdict

ChainForge is a strong recommendation for technical individuals and teams that want open-source, provider-spanning prompt experiments with local control and no software fee. Choose it for its comparative evaluation depth and extensibility; look elsewhere if you need a fully hosted, automated and operationally forgiving product.