Kiln
What it is
Kiln is an AI workbench for teams building and optimizing AI systems, with tools for evaluation, fine-tuning, and agent work. Users can describe an evaluation goal to generate prompts and synthetic datasets, then benchmark judges against human ratings. Kiln supports composing agents and subagents, inspecting their tool calls, and evaluating each level of an agent hierarchy. It works with MCP servers and providers including AWS Bedrock, Azure OpenAI, Vertex AI, OpenAI, Anthropic, Fireworks, Gemini, Ollama, and OpenAI-compatible endpoints. Datasets, traces, and evaluations sync to Git; the enterprise page names GitHub Enterprise, GitLab, Bitbucket, and Azure DevOps. Prompts, outputs, ratings, and fine-tuning jobs are saved as JSON files on the user's filesystem, and provider calls go directly from the user's machine to the chosen provider. Kiln says it can run offline with Ollama, vLLM, Llama.cpp, or another compatible local server. The free Individual plan includes a Python library, source-available desktop app, local datasets with Git sync, rate-limited standard models, and community support.
For and against
For
- Generates evaluation prompts and synthetic datasets
- Supports agent hierarchies and tool-call inspection
- Stores work as JSON files on the user's filesystem
- Can run offline with listed local servers
Against
- Individual standard models are rate limited
- Desktop app collects UI analytics through PostHog
The verdict
Kiln brings evaluation, agent development, and optimization into a toolset with local data storage and multiple provider options.
Who it suits
Kiln is aimed at engineers, data scientists, product managers, QA specialists, and subject matter experts working on AI systems. Its local storage and offline options may suit teams that want to keep their work on their own machines.
Plans and prices
Individual
Open Source Python Library · Desktop App (source-available) · Local datasets with git sync · Kiln Pro standard models, rate limited
Read 7 Oct 2026 · kiln.tech
Enterprise
Everything in Team · SSO / SAML · Annual contracts & procurement support · SLA & priority support
Read 7 Oct 2026 · kiln.tech
Team
Everything in Individual · Kiln Pro enhanced models, higher limits · Automatic agent optimization · Priority access to new features
Read 7 Oct 2026 · kiln.tech
On the record
- Agents
- Kiln supports composing agents and subagents, inspecting tool calls, and evaluating each level of an agent hierarchy.kiln.tech · read 7 Oct 2026
- Collaboration
- Kiln automatically syncs datasets, traces, and evals to Git, and its enterprise page names GitHub Enterprise, GitLab, Bitbucket, and Azure DevOps.kiln.tech · read 7 Oct 2026
- Company
- The website footer identifies Kiln AI as Chesterfield Laboratories Inc.kiln.tech · read 7 Oct 2026
- Enterprise controls
- Enterprise services can include SSO, VPC deployment, and telemetry controls.kiln.tech · read 7 Oct 2026
- Evaluation
- Kiln can generate evaluation prompts and synthetic datasets from a description of the goal, then benchmark judges against human ratings.kiln.tech · read 7 Oct 2026
- Integrations
- Kiln supports MCP servers and lists providers including AWS Bedrock, Azure OpenAI, Vertex AI, OpenAI, Anthropic, Fireworks, Gemini, Ollama, and OpenAI-compatible endpoints.kiln.tech · read 7 Oct 2026
- Intended users
- Kiln describes its audience as engineers, data scientists, product managers, QA, and subject matter experts working on AI systems.kiln.tech · read 7 Oct 2026
- Local data
- Kiln stores prompts, outputs, ratings, and fine-tuning jobs as JSON files on the user's filesystem, while provider API calls go directly from the user's machine to the chosen provider.kiln.tech · read 7 Oct 2026
- Offline use
- Kiln says it can run fully offline with Ollama, vLLM, Llama.cpp, or another OpenAI-compatible local server.kiln.tech · read 7 Oct 2026
- Product
- Kiln describes itself as an AI workbench for teams to build and optimize AI systems, including evals, fine-tuning, and optimization.kiln.tech · read 7 Oct 2026
- Security
- Kiln says its Python library is MIT-licensed and its source-available desktop app has public CI builds and checksums for every release.kiln.tech · read 7 Oct 2026
- Support
- The Individual plan includes community support, Team includes email support, and Enterprise includes SLA and priority support.kiln.tech · read 7 Oct 2026
- Telemetry
- The desktop app collects UI analytics through PostHog, while the Python library sends no analytics; the company says telemetry can be disabled for enterprise contracts.kiln.tech · read 7 Oct 2026
