- Home
- Mutation Testing Tools
- Cosmic Ray
Cosmic Ray
What it is
Cosmic Ray is a free mutation-testing tool for Python 3 projects. It changes source code in small ways and runs a test suite for each change to check whether the tests catch the altered behavior. This gives teams a way to assess whether tests verify behavior, beyond knowing which lines were executed. It supports unittest, pytest and nose, and includes cr-report for session summaries and cr-html for HTML reports showing mutations.
A TOML configuration specifies the modules to mutate, test command, timeout, exclusions and distributor. The local distributor runs tests serially; the HTTP distributor can send work to local or remote workers for parallel execution, with each worker needing its own copy of the code. Users can add distributor and mutation-operator plugins. Cosmic Ray changes code directly on disk, so its documentation advises committing changes first. Runs can take a long time because the suite is executed for each mutation, and the full configuration options are not currently well documented. Tests in a module shared with production code may also be mutated unless separated or excluded.
For and against
For
- Free software identified as MIT licensed.
- Supports unittest, pytest and nose.
- Can distribute work to local or remote workers.
- Provides session summaries and HTML mutation reports.
- Users can add mutation-operator plugins.
Against
- Mutation runs can take a long time.
- The tool changes source code directly on disk.
- Full configuration options are not currently well documented.
- Tests sharing production modules may also be mutated.
The verdict
Cosmic Ray offers configurable mutation testing with serial or distributed execution and report tools. Its direct edits to code and potentially long runs make committing changes beforehand and planning runtime important.
Who it suits
Cosmic Ray suits Python project teams who want to check whether their test suites detect changes in behavior. It supports unittest, pytest and nose workflows.
On the record
- Code handling
- Cosmic Ray applies mutations directly to code on disk, and the documentation advises committing changes first because a mutation might not be correctly undone after an exception or crash.cosmic-ray.readthedocs.io · read 3 Oct 2026
- Configuration limitation
- The documentation says the full set of configuration options is not currently well documented.cosmic-ray.readthedocs.io · read 3 Oct 2026
- Custom extensions
- Users can provide their own distributor and mutation operator plugins.cosmic-ray.readthedocs.io · read 3 Oct 2026
- Execution modes
- The local distributor runs tests serially on the local machine, and the HTTP distributor sends work to local or remote workers that can run tests in parallel.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Extensibility
- Users can extend Cosmic Ray with their own mutation operators exposed through plugins.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Install
- The tutorial says to install Cosmic Ray with `pip install cosmic-ray`.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Intended users
- The documentation presents Cosmic Ray for Python projects whose teams want to check whether their test suites meaningfully verify code behavior.cosmic-ray.readthedocs.io · read 3 Oct 2026
- License
- The project repository identifies the software as MIT licensed.github.com · read 4 Oct 2026
- Maker
- The project documentation credits Sixty North AS.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Mutation filters
- Included filters can skip mutations by operator name, a `# pragma: no mutate` comment, or whether the mutated line changed relative to a Git branch.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Notable limitation
- Cosmic Ray may mutate test code if it shares a module with the production code under test; the documentation recommends keeping tests separate or excluding their modules.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Parallel execution
- The HTTP distributor sends mutation and test requests to workers that can run locally or remotely; each worker needs its own copy of the code under test.cosmic-ray.readthedocs.io · read 3 Oct 2026
- Purpose
- Cosmic Ray is a mutation testing tool for Python 3 that makes small changes to production code and runs the test suite against each change.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Reports
- Cosmic Ray provides `cr-report` for session summaries and `cr-html` for generating an HTML report that shows the mutation used.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Runtime consideration
- Mutation testing can take a long time because the tool runs the test suite for each mutation; the documentation gives an example where 1,000 mutations and a 10-second test suite take about 2.7 hours.cosmic-ray.readthedocs.io · read 3 Oct 2026
- Runtime limit
- A configured timeout ends a mutant test that runs too long, and Cosmic Ray marks that mutant incompetent.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Target users
- The documentation describes Cosmic Ray as a tool for Python projects and says it has been used on projects ranging from assemblers to oil exploration software.cosmic-ray.readthedocs.io · read 4 Oct 2026
- Test commands
- A TOML configuration specifies the modules to mutate, test command, timeout, exclusions, and distributor.cosmic-ray.readthedocs.io · read 3 Oct 2026
