Rogue

Rogue is a self-hosted platform for evaluating AI agents and testing their reliability and security. Automatic evaluations check agent behavior against business policies and expected outcomes, then produce pass/fail reports with reasoning. Red-team coverage includes more than 75 vulnerabilities across 12 security categories and attack techniques such as encoding, social engineering, and injection. Vulnerabilities receive CVSS-based risk scores from 0 to 10. Rogue maps testing to frameworks including OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, ISO/IEC 42001, the EU AI Act, GDPR, and OWASP API Top 10. It provides a server, terminal interface, and non-interactive CLI for CI/CD pipelines. Supported protocols include A2A over HTTP, MCP over SSE or Streamable HTTP, and direct Python calls. OpenAI, Anthropic, and Google models are available through LiteLLM; an LLM API key is required. Scans can use a random seed for reproducibility, and reports export to Markdown, CSV, or JSON. Basic scans use five curated vulnerabilities and six attacks; full scans use more than 75 vulnerabilities and 40 or more attacks.

Rogue combines policy-based evaluation with red-team scans, framework mapping, and exportable reports. Its free use is for personal and internal purposes; commercial hosting requires licensing.

Rogue is presented for security teams and developers building or deploying AI agents. It may suit teams needing regression testing, security audits, or compliance reporting.

Rogue is a self-hosted platform for evaluating AI agents and red-teaming them for reliability and security. It is aimed at developers and security teams that need repeatable checks, release-gate automation, or compliance-oriented evidence. The combination of broad attack coverage, framework mapping, and exportable results makes it a strong fit for internal agent programs, while its licensing terms rule out some hosted-business models.

Overview

Rogue evaluates an agent against business policies and expected behavior, returning pass/fail findings with reasoning rather than only a raw score. That makes it useful when a team must explain why an agent failed a control or compare behavior across releases. It is automated, supports custom tests, and offers continuous monitoring, so it can serve both scheduled assurance work and recurring regression checks.

The project is self-hosted. Getting started requires uvx, Python 3.10 or newer, and an API key for an OpenAI, Anthropic, or Google model. Rogue accesses those providers through LiteLLM. Teams that cannot supply an LLM key, or that want a turnkey hosted service, should look elsewhere.

Key features

Red-team scans cover 75+ vulnerabilities in 12 security categories and 20 attack techniques. Encoding, social engineering, injection, semantic, and technical attacks are represented, with targets including A2A agents, MCP agents, and Python agents. Basic scans use five curated vulnerabilities and six attacks; full scans expand to 75+ vulnerabilities and 40+ attacks. Basic mode is a practical, narrower check, but a serious security review needs the broader full-scan scope.

Each vulnerability receives a CVSS-based risk score from 0 to 10, giving teams a consistent way to prioritize remediation. Testing maps to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, ISO/IEC 42001, the EU AI Act, GDPR, and OWASP API Top 10. That breadth is valuable for organizations that must connect technical findings to governance or regulatory work.

Rogue includes a server, terminal interface, and non-interactive CLI. The CLI is the important option for CI/CD pipelines, where evaluations can become repeatable release checks. Reports export as Markdown, CSV, or JSON for engineering, audit, and ticketing workflows. A random seed can make scans reproducible, which helps compare runs over time. Runtime integrations described for the wider AI AppSec offering include LangChain, CrewAI, AutoGen, custom builds, SIEM systems, and webhooks such as Splunk, Datadog, and PagerDuty.

Pricing

The Personal and internal use plan costs free and is free for personal and internal use. It is the sensible starting point for individual builders and teams assessing agents they operate themselves, but it does not grant permission to sell Rogue as a hosted service.

Commercial hosting has custom pricing and requires licensing; contact [email protected]. This matters to consultancies, SaaS vendors, and platform operators planning to offer Rogue-backed scanning to customers. The free plan has no published seat or scan quota, so organizations should confirm operational limits before standardizing on it.

Platforms

Rogue runs self-hosted rather than as a listed web, desktop, or mobile service. Supported agent protocols are A2A over HTTP, MCP over SSE or Streamable HTTP, and direct Python function calls. OpenAI, Anthropic, and Google models are supported through LiteLLM.

Who it's for

Choose Rogue if you build or deploy AI agents and need policy evaluation, adversarial testing, reproducible regression scans, or evidence aligned with security and compliance frameworks. It is particularly suitable for security teams and developers integrating checks into CI/CD. It is less suitable for buyers seeking a managed interface, a published commercial tier, or permission to resell hosted access without a separate license.

Pros and cons

• Pros: 75+ vulnerabilities, 40+ full-scan attacks, and multiple attack categories provide meaningful breadth.
• Pros: CVSS 0–10 scoring and seven named governance and security frameworks help prioritize and document risk.
• Pros: Server, terminal, and CI/CD CLI interfaces support both interactive work and automation.
• Pros: Markdown, CSV, and JSON exports plus seeded runs support reporting and repeatable comparisons.
• Cons: Basic scans cover only five vulnerabilities and six attacks, so they are not equivalent to full coverage.
• Cons: Self-hosting, Python tooling, and a required provider API key add setup responsibility.
• Cons: Commercial hosting requires licensing, limiting hosted resale and some customer-facing deployments.

Alternatives

For a broader directory of choices, see AI Red Teaming Tools.

• F5 BIG-IP APM is a paid access-management product with a free trial and wide platform support, better suited to access control than agent red teaming.
• NVADER offers a web-based freemium option with a free tier for one app, making it preferable when hosted access matters.
• RedLens AI provides hosted and API deployment with a free allowance of 25 scans per month and five per day.
• AgentSeal is a freemium choice with 30 basic probes, MCP registry, Guard, and Watch features; choose it when those packaged controls fit better than Rogue's self-hosted workflow.
• Confident AI is a freemium API, self-hosted, and web platform with a free tier limited to two seats, one project, and five test runs weekly.
• Darkhunt AI Security offers free credits and full platform access, useful for teams prioritizing a hosted platform with community support.
• Giskard provides an open-source library and local deployment for basic LLM vulnerability scanning and RAG evaluation.
• OpenSecureAI Scanner has a $49-per-month Pro plan with 50,000 API requests and hosted remediation features, suited to buyers wanting a published paid tier.

Verdict

Rogue is a strong choice for security-conscious developers and internal security teams that want self-hosted agent evaluation, substantial red-team coverage, framework mapping, and CI/CD-ready reports. Choose it primarily for the breadth and repeatability of its scans. Look elsewhere if you need a managed hosted product, clearly published commercial limits, or unrestricted rights to offer scanning as a paid service.