Noveum

Noveum is a platform for evaluating AI agents used in production chat, voice, and workflow settings. NovaTrace records language-model calls, tool use, retrieval, and agent steps, with tokens, cost, and latency shown on spans. NovaEval scores traces using calibrated scorers across 15+ categories, including voice and audio. For calls that fail, NovaPilot can backtest candidate fixes, re-simulate them end to end, and deliver validated changes as pull requests for human review.

The tracing SDK has native integrations for LangChain, LangGraph, LiveKit, Pipecat, and CrewAI, plus compatibility with OpenAI, Anthropic, and OpenTelemetry. Open-source tracing SDKs support Python and TypeScript. Deployment options include managed cloud, VPC, BYO ClickHouse, and on-premise, with Kubernetes and Helm options also described. Plans include a free tier and Pro at $69/mo with a 7-day trial. Monthly credits reset each billing cycle without rollover; purchased add-on credits do not expire. Runtime guardrails are marked beta or coming soon.

Noveum combines trace capture, evaluation, and fix validation for production agent workflows. Its free plan and deployment choices offer multiple ways to use the platform, while credit rollover limits are worth noting.

Noveum suits teams operating production chatbots, voice agents, or multi-agent workflows that need to trace and evaluate agent activity. Its deployment options include managed cloud and self-hosted approaches.

Noveum is a platform for evaluating production chat, voice, and workflow agents. It is best suited to teams that need to trace agent activity, score results, and check proposed fixes. Its connected workflow is the main draw; credit limits and plan terms matter as usage grows.

Overview

Noveum combines production tracing, evaluation, and fix validation. NovaTrace records model calls, tool use, retrieval, and agent steps, with tokens, cost, and latency attached to spans. NovaEval scores traces across more than 15 categories, including voice and audio. NovaPilot backtests candidate fixes on failing calls, re-simulates them end to end, and submits validated fixes as pull requests for human review.

That sequence gives teams a way to move from diagnosing agent behavior to reviewing proposed corrections within one platform. Noveum is less suitable if runtime guardrails are a current requirement: NovaGuard is marked beta or coming soon.

Key features

Trace and score agent activity

Native tracing integrations cover LangChain, LangGraph, LiveKit, Pipecat, and CrewAI, with compatibility also listed for OpenAI, Anthropic, and OpenTelemetry. Open-source tracing SDKs support Python and TypeScript. The breadth is useful for teams operating different text, voice, and multi-agent stacks; the practical value depends on whether their frameworks fit those integrations.

Evaluation combines hybrid methods, tool-call checks, safety evaluations, and regression runs. The 15-plus scoring categories and voice suite give teams more than basic trace inspection, while span-level cost and latency help put observed behavior in operational context.

Validate fixes with human review

NovaPilot takes failing calls through backtesting and end-to-end re-simulation before delivering candidate fixes as pull requests. That keeps human review in the loop rather than presenting automated changes as final. Teams looking only for trace capture may not need this broader correction workflow.

Pricing

Noveum has a free plan, recurring subscriptions, one-time add-on credit packs, and custom-priced Enterprise service. Monthly credits reset each billing cycle and do not roll over; purchased add-on credits never expire. That distinction makes packs useful for occasional bursts, but means subscription credits need to be used within their cycle.

The free tier gives small teams room to try the workflow, but its 3-member cap, 1M monthly spans, and 30-day retention constrain sustained or collaborative use. Pro is the entry subscription at $69/mo, with a 7-day trial and larger quotas; Growth and Max add capacity, storage, seats, and retention for teams with heavier workloads. Scale raises those limits substantially and removes the member cap, while Enterprise is the fit for organizations needing custom retention and unlimited listed quotas. Support advances from community on Free to priority on Growth and above, then dedicated customer success for Enterprise. SSO/SAML is specified for Scale and above.

Platforms

Noveum supports API, Linux, web, and self-hosted use. Deployment choices include managed cloud, VPC, BYO ClickHouse, and on-premise; Kubernetes and Helm deployment options are also described. This range suits teams with deployment constraints, though the right fit depends on their infrastructure requirements.

The maker says GDPR and SOC 2 Type II are in progress. Enterprise deployments include encryption in transit and at rest and fine-grained access controls; the homepage also lists SSO/SAML, RBAC, and audit logging. Teams should note that the pricing page ties SSO/SAML to Scale and above.

Who it's for

Noveum is a strong fit for teams running production chatbots, voice agents, or multi-agent workflows that need to connect traces, scoring, and fix review. It is less compelling for a team seeking only a local evaluation library or immediate runtime guardrails.

Pros and cons

• Pro: Tracing, evaluation, and fix validation share one workflow, so teams can connect observed failures with reviewed corrections.
• Pro: Voice and audio scoring, tool-call checks, safety evaluation, and regression runs address more than text-only quality checks.
• Pro: Managed and self-hosted deployment options give teams meaningful infrastructure choices.
• Con: Free usage is capped at 2.5K credits and 1M spans per month, with 3 members and 30-day retention.
• Con: Subscription credits expire at each billing-cycle reset, so unused monthly capacity cannot be carried forward.
• Con: NovaGuard runtime guardrails are not yet a settled option, as they are marked beta or coming soon.

Alternatives

AI Agent Evaluation Tools is a useful directory for comparing products in the category.

• Arklex is the better match for teams seeking an open-source agent testing framework with a free plan and API, Linux, and self-hosted platforms.
• Future AGI AI Evaluation SDK fits teams prioritizing Apache 2.0 licensing and self-hosting through Docker, Python SDK, or Node SDK, with a free plan.
• AgentClash is worth considering for a free evaluation workflow capped at 25 runs per month, with up to four models per run and seven-day replay retention.
• Braintrust offers a free Starter plan with 1 GB processed data, 10,000 scores, 14-day retention, and unlimited users, projects, and datasets.
• DeepEval is a free, Apache 2.0-licensed local and CI/CD test runner for teams that want an open-source evaluation framework.
• Galileo is an alternative with a Pro plan at $100 a month billed yearly, including 50,000 traces per month, Standard RBAC, and dedicated Slack support.
• Giskard provides a free, open-source library with local deployment and basic LLM vulnerability scanning and RAG evaluation.
• Google Cloud Agent Evaluation is another alternative.

Verdict

Choose Noveum if your team operates production agents and wants tracing, evaluation, and human-reviewed fix validation tied together, with deployment options beyond managed cloud. Look elsewhere if you need runtime guardrails now, or if monthly credit rollover is important; its free plan and add-on packs offer flexibility, but neither removes the recurring subscription limits.