Deepchecks

No phone app

6.4No. 16 of 24
in Model Monitoring Software

Recognised40% of the score25
Phone app26% of the score0
Documented20% of the score91
Free plan14% of the score100

Free plan: Yes
Runs on: api, Linux, self-hosted, Web

Summary

Deepchecks LLM Evaluation is a platform for testing, observing and monitoring AI systems in production. Its evaluation tools include automated scoring, version comparisons, configurable and ready-made properties, and golden set management. Production monitoring applies checks to help assess whether LLMs continue to perform consistently. Filtering and drill-down tools help investigate application steps and root causes. Deployment choices include multi-tenant SaaS, virtual private cloud on GCP or Azure, bare metal and AWS-managed deployment. For the AWS-managed option, Deepchecks says in-app data and artifacts remain within the environment perimeter. Listed integrations include pytest, Apache Airflow, ZenML, CML, Amazon Bedrock and SageMaker AI. The site lists SOC 2 Type 2, GDPR and HIPAA compliance, single sign-on and AWS GovCloud support. The free Basic plan includes up to 3 seats, 1 AI application, up to 5K DPUs per month, 3 months of data retention and unlimited prompt-based metrics. DPUs measure SaaS usage, and plans have fixed monthly allocations. Basic has a free trial; paid plans start at $159.

Who it is for

It suits teams evaluating and monitoring LLM applications in production, especially those needing configurable deployment options or integrations. The free Basic plan is limited to one AI application and up to 3 seats.

What is good

Automated scoring and version comparisons are available.
Offers SaaS, private cloud, bare metal and AWS-managed deployment.
Free Basic plan includes up to 5K DPUs monthly.
Lists SOC 2 Type 2, GDPR and HIPAA compliance.

What to know first

Basic allows only 1 AI application.
Basic retains data for 3 months.
Each plan has a fixed monthly DPU allocation.

Verdict

Deepchecks LLM Evaluation combines evaluation and production monitoring with several deployment options. The free Basic plan has defined seat, application, DPU and retention limits; paid plans start at $159.

Deepchecks plans and pricing

All plans

Basic Free Up to 3 seats · 1 AI application · Up to 5K DPUs/month · 3 months data retention · Unlimited prompt-based metrics deepchecks.com · 28 Sept 2026

Scale Not published 5 seats · 3 AI applications · 20K DPUs/month · Premium support · Premium compliance · Guided platform onboarding deepchecks.com · 28 Sept 2026

Enterprise Not published Custom seats and AI applications · Custom DPUs/month · Enterprise-grade security · Enterprise support package · Dedicated customer success team deepchecks.com · 28 Sept 2026

Compared on model monitoring software

Free plan: Yesdeepchecks.com
Drift monitoring: Yesdeepchecks.com
Model performance metrics: Yesdeepchecks.com
Data quality checks: Yesdeepchecks.com
Bias monitoring: Yesdeepchecks.com
Alert channels: email, Slack, webhooksdeepchecks.com
Deployment options: hybriddeepchecks.com
Included model limit: 10 modelsdeepchecks.com

Facts

Product: Deepchecks LLM Evaluation is an AI testing, observability, and monitoring platform for AI systems in production.deepchecks.com · 28 Sept 2026
Evaluation: The platform offers automated scoring, version comparison, custom and off-the-shelf properties, and golden set management.deepchecks.com · 28 Sept 2026
Monitoring: Deepchecks describes production monitoring as a way to apply checks to help ensure LLMs consistently perform.deepchecks.com · 28 Sept 2026
Debugging: The product offers filtering and drill-down to investigate application steps and find root causes.deepchecks.com · 28 Sept 2026
Deployment: Deployment options listed include multi-tenant SaaS, virtual private cloud on GCP or Azure, bare metal, and AWS-managed deployment.deepchecks.com · 28 Sept 2026
Security: The site lists SOC 2 Type 2, GDPR, HIPAA compliance, single sign-on, and AWS GovCloud support.deepchecks.com · 28 Sept 2026
Data locality: For its AWS-managed deployment, the site says in-app data and artifacts do not leave the environment perimeter.deepchecks.com · 28 Sept 2026
Integrations: The integrations page describes Deepchecks integrations with pytest, Apache Airflow, ZenML, and CML.deepchecks.com · 28 Sept 2026
AWS integration: The AWS-managed option is described as integrating with Amazon Bedrock and SageMaker AI.deepchecks.com · 28 Sept 2026
Trial: The pricing page offers a free trial for the Basic plan.deepchecks.com · 28 Sept 2026
Pricing limits: The pricing page says DPUs are the SaaS usage measure and that each plan includes a fixed monthly DPU allocation.deepchecks.com · 28 Sept 2026
Company: Deepchecks says it was founded by a group with machine-learning research and applied machine-learning experience.deepchecks.com · 28 Sept 2026