- Home
- A/B Testing Tools
- Experimently
Experimently
What it is
Experimently is an open-source platform for A/B testing and feature flags that runs on infrastructure you manage. Its Free Preview costs nothing when self-hosted on your AWS account and includes all features; the platform is Apache-2.0 licensed and has no hosted tier. It supports A/B and multivariate tests, sequential testing, CUPED, Bayesian testing, multi-armed bandits, split URL tests, and LLM/model evaluation. Rollout controls include sticky assignments, staged releases, mutual-exclusion groups, a global holdout, and automatic rollback based on error-rate and latency thresholds. Targeting has more than 20 operators, including semantic version comparison, geo-distance, time windows, JSON paths, and array operations. Analysis offers frequentist and Bayesian methods, sequential mSPRT testing, and segment breakdowns with Bonferroni correction. SDKs are listed for many languages and mobile platforms, but are not yet available through package registries; installation instructions are in each guide. Deployment options include Docker Compose, Kubernetes with Helm, or an AWS CDK app. Assignment data stays on your infrastructure.
For and against
For
- Free Preview includes all features when self-hosted on AWS.
- Supports staged rollouts and automatic rollback thresholds.
- Targeting includes more than 20 operators.
- Offers frequentist, Bayesian, and sequential analysis methods.
- Deployment options include Docker Compose, Kubernetes, and AWS CDK.
Against
- There is no hosted tier.
- SDKs are not yet available through package registries.
- Users must deploy on their own infrastructure.
The verdict
Experimently brings a wide range of testing, rollout, targeting, and analysis methods to self-managed infrastructure. Its free preview is self-hosted, and SDK installation currently relies on instructions in each guide rather than package registries.
Who it suits
It may suit teams seeking experimentation and feature-flag tools they can run on their own infrastructure. It also fits developers looking for multiple deployment approaches and SDK options, provided they can install SDKs from the individual guides.
Plans and prices
Free Preview
Self-hosted on your AWS account · all features included · unlimited A/B experiments
Read 14 Oct 2026 · getexperimently.com
On the record
- A/B and multivariate testing
- The product documentation covers A/B, multivariate, sequential, split-URL, Bayesian and bandit experiments.getexperimently.com · read 14 Oct 2026
- Compliance reporting
- The documentation provides export functions for SOC 2 and ISO 27001 reports and describes HIPAA-related encryption, retention and BAA features.getexperimently.com · read 14 Oct 2026
- CUPED
- CUPED variance reduction is documented as a method for reducing variance using pre-experiment covariates.getexperimently.com · read 14 Oct 2026
- Data handling
- The maker says assignment data stays on the user's own infrastructure and that there is no hosted tier.getexperimently.com · read 5 Oct 2026
- Data integrations
- Documentation lists AWS, Snowflake, BigQuery, Redshift, Databricks, ClickHouse and MySQL connectors.getexperimently.com · read 14 Oct 2026
- Deployment
- Users can deploy with Docker Compose, Kubernetes using a Helm chart, or an AWS CDK app targeting ECS Fargate, Aurora PostgreSQL, and ElastiCache in their own AWS account.getexperimently.com · read 5 Oct 2026
- Deployment options
- Customers can run published containers with Docker Compose, deploy to Kubernetes with Helm, or use an AWS CDK deployment.getexperimently.com · read 14 Oct 2026
- Documentation depth
- The documentation site includes quick starts, API schemas, SDK guides, integration references, security topics, deployment instructions and statistics references.getexperimently.com · read 14 Oct 2026
- Experiment controls
- The platform supports sticky assignment, staged rollouts, mutual-exclusion groups, global holdouts and automatic rollback triggered by error-rate or latency thresholds.getexperimently.com · read 14 Oct 2026
- Experiments and rollouts
- It supports sticky assignment, staged rollouts, mutual-exclusion groups, a global holdout, and automatic rollback based on error-rate and latency thresholds.getexperimently.com · read 5 Oct 2026
- Independent maintainer
- PyPI identifies amarkanday as the sole maintainer of the placeholder experimently package.pypi.org · read 14 Oct 2026
- Independent package maturity
- PyPI classifies the package as development status 1, Planning, and lists Python 3.9 or newer as a requirement.pypi.org · read 14 Oct 2026
- Independent performance target
- The project article sets a target of under 50 ms at p99 for feature-flag evaluation and more than 10,000 assignments per second.medium.com · read 14 Oct 2026
- Independent planned architecture
- The project article describes an AWS architecture using ECS/Fargate, Lambda, Aurora PostgreSQL, DynamoDB, ElastiCache, S3, Kinesis, EventBridge, OpenSearch, Glue and Athena.medium.com · read 14 Oct 2026
- Independent project date
- The product-management article describing the platform was published March 28, 2025.medium.com · read 14 Oct 2026
- Independent project origin
- Ashish Markanday wrote that building the platform was a personal project intended to create a production-quality experimentation system with AI assistance in roughly three to four months.medium.com · read 14 Oct 2026
- Independent risk: complexity
- The project article identifies high-throughput real-time evaluation, data integrity, integration complexity and statistical validity as material implementation risks.medium.com · read 14 Oct 2026
- Integration API
- Integration endpoints support Jira, Salesforce and GitHub, including webhook receivers and role-based permissions.getexperimently.com · read 14 Oct 2026
