- Home
- Test Flakiness Management Software
- FlakyTest.dev
FlakyTest.dev
Ljubljana, Slovenia
What it is
FlakyTest.dev is a web service for Python teams managing unreliable test suites. Its pytest plugin tracks tests and sends results when the FLAKYTEST_SECRET_TOKEN environment variable is set. Test ownership can direct flaky-test notifications to the person who wrote a test. From the interface, users can mute a test without changing code or opening a pull request; the test keeps running and logging errors without failing the CI build. The service shows run history for individual tests and displays standard output and standard error for failures. Run statistics help identify tests that slow builds, while flake detection and a quarantine workflow support ongoing management. The setup guide documents GitHub Actions configuration with the same repository secret. FlakyTest.dev has an OSS plan at no cost for personal projects, nonprofits, and open source projects. Business includes a 60-day free trial, with later pricing undetermined.
For and against
For
- Pytest tracking and test management
- Mute tests without code changes
- Run history and failure debug output
- Ownership-based flaky-test notifications
- Flake detection and quarantine workflow
Against
- Currently focused on Python
- Business pricing after trial is undetermined
- Results are sent when a secret variable is set
- Security policy cannot guarantee absolute security
The verdict
FlakyTest.dev gives Python teams tools to track flaky tests, inspect failures, and keep muted tests from breaking CI builds. The free OSS plan covers qualifying projects, while Business pricing is decided after its trial.
Who it suits
It is aimed at Python teams dealing with unreliable tests and developers using pytest. The product currently focuses on Python projects.
Plans and prices
OSS
Personal projects · non profits · open source projects
Read 4 Oct 2026 · flakytest.dev
Business
60 day free trial · pricing after trial to be determined
Read 4 Oct 2026 · flakytest.dev
On the record
- Data sent
- The pytest plugin sends test results to FlakyTest when the FLAKYTEST_SECRET_TOKEN environment variable is set.flakytest.dev · read 4 Oct 2026
- Debug details
- For failing tests, it shows debug information including standard output and standard error.flakytest.dev · read 4 Oct 2026
- Integrations
- The setup guide documents GitHub Actions configuration using a repository secret named FLAKYTEST_SECRET_TOKEN.flakytest.dev · read 4 Oct 2026
- Intended users
- The maker describes FlakyTest as a tool for Python teams dealing with unreliable test suites.flakytest.dev · read 4 Oct 2026
- Language limit
- The maker says the product currently focuses on Python projects and plans to support other languages in the future.flakytest.dev · read 4 Oct 2026
- Maker background
- Anže describes himself as a software engineer and indie maker with over ten years of experience shipping products for Silicon Valley startups and growing development teams.flakytest.dev · read 4 Oct 2026
- Mute tests
- Users can mute a test in the UI without code changes or pull requests, while the test continues running and logging errors without failing the CI build.flakytest.dev · read 4 Oct 2026
- Purpose
- FlakyTest is a place to manage, monitor, and report on test suite performance.flakytest.dev · read 4 Oct 2026
- Run statistics
- Run statistics can help identify tests that make builds slow.flakytest.dev · read 4 Oct 2026
- Security and privacy
- The privacy policy says the service uses commercially acceptable means to protect personal data but cannot guarantee absolute security.flakytest.dev · read 4 Oct 2026
- Support
- The documentation directs users needing help, support, or feedback to email the maker.flakytest.dev · read 4 Oct 2026
- Test framework
- Its pytest plugin enables test tracking and management for Python projects.flakytest.dev · read 4 Oct 2026
- Test history
- The product shows all runs for an individual test.flakytest.dev · read 4 Oct 2026
- Test ownership
- It uses test ownership to direct flaky test notifications to the person who wrote the test.flakytest.dev · read 4 Oct 2026
