- Home
- AI Prompt Generators
- PromptPex
On the recordRead 8 Oct 202619 facts
PromptPex
On the record
- Automatic rule extraction
- PromptPex extracts output rules written in natural language in a prompt, such as a requirement to format output as JSON.github.com · read 8 Oct 2026
- CLI and Docker
- PromptPex is packaged as an npm command-line tool and can also be run in Docker.microsoft.github.io · read 8 Oct 2026
- Configurable generation
- Test generation can be configured with scenarios, test expansions, and custom instructions.microsoft.github.io · read 8 Oct 2026
- Deployment
- The project README gives a Docker command for running PromptPex, and the setup guide describes a local web interface.github.com · read 8 Oct 2026
- Feedback
- The transparency note invites feedback and collaboration by email and says the team will update the repository with mitigations for reported or identified issues.microsoft.github.io · read 8 Oct 2026
- Integrations
- PromptPex documents exports and integrations for OpenAI Evals, Azure OpenAI Stored Completions, GitHub Models Evals, and GitHub Actions.microsoft.github.io · read 8 Oct 2026
- Intended audience
- The project describes its intended users as developers who keep prompts in their codebase.microsoft.github.io · read 8 Oct 2026
- Intended use
- The released open-source library is intended for research, and the maker says it should not be used in downstream applications without additional evaluation of robustness, safety, harm, and bias.microsoft.github.io · read 8 Oct 2026
- Limitations
- The maker says vague prompts can produce suboptimal results and rule extraction can vary in accuracy, especially with complex or idiomatic language.microsoft.github.io · read 8 Oct 2026
- Maker
- The documentation footer identifies Microsoft as the maker of PromptPex.microsoft.github.io · read 8 Oct 2026
- Model evaluation
- PromptPex can evaluate prompt performance across models, and uses an LLM to assess whether model outputs meet specified requirements.github.com · read 8 Oct 2026
- Model providers
- The setup guide says users can configure OpenAI, Azure OpenAI, or Ollama credentials.microsoft.github.io · read 8 Oct 2026
- Prompt formats
- The CLI supports Markdown or text templates, Prompty files, and GitHub Models prompt YAML files.microsoft.github.io · read 8 Oct 2026
- Purpose
- PromptPex is a tool for developers to explore and test AI model prompts by treating prompts as functions and generating test inputs.microsoft.github.io · read 8 Oct 2026
- Rule extraction
- It can extract output rules written in natural language in a prompt.github.com · read 8 Oct 2026
- Support
- The transparency note invites feedback and reports of unexpected or offensive behavior by email to listed project contacts.microsoft.github.io · read 8 Oct 2026
- Supported model providers
- The CLI setup lists OpenAI, Azure OpenAI, GitHub Models, and Ollama among supported LLM providers.microsoft.github.io · read 8 Oct 2026
- Test generation
- It generates unit test cases designed to check whether a prompt makes a model follow its output rules.github.com · read 8 Oct 2026
