- Home
- LLM Evaluation Tools
- EvalPlus
On the recordRead 11 Oct 202618 facts
EvalPlus
On the record
- Benchmarks
- The framework supports HumanEval, MBPP, and EvalPerf evaluation.github.com · read 11 Oct 2026
- Docker
- The README documents running code evaluation in Docker.github.com · read 5 Oct 2026
- Efficiency evaluation
- EvalPerf evaluates the efficiency of LLM-generated code using performance-exercising tasks and test inputs.github.com · read 11 Oct 2026
- EvalPerf leaderboard
- The EvalPlus site provides a leaderboard for EvalPerf code efficiency results.evalplus.github.io · read 5 Oct 2026
- Execution
- The README provides Docker-based examples for running code evaluation in a container.github.com · read 11 Oct 2026
- HumanEval+
- HumanEval+ expands the original HumanEval tests by 80 times.evalplus.github.io · read 11 Oct 2026
- Installation
- EvalPlus can be installed from its GitHub repository or as a stable release using pip.github.com · read 11 Oct 2026
- Intended audience
- The project is intended for evaluating LLM performance on code-related tasks.evalplus.github.io · read 5 Oct 2026
- Leaderboard
- The project links leaderboards for EvalPlus and EvalPerf benchmarks.evalplus.github.io · read 11 Oct 2026
- License
- The repository provides the software under the Apache License 2.0, with files under evalplus/eval additionally complying with the MIT License.github.com · read 11 Oct 2026
- MBPP+
- MBPP+ expands the original MBPP tests by 35 times.evalplus.github.io · read 11 Oct 2026
- Model backends
- Documented backends include Hugging Face, vLLM, OpenAI-compatible servers, OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Ollama, and Intel Gaudi.github.com · read 11 Oct 2026
- Operating system limit
- The README labels EvalPerf as *nix only.github.com · read 11 Oct 2026
- Platform limit
- The README labels EvalPerf code efficiency evaluation as *nix only.github.com · read 5 Oct 2026
- Purpose
- EvalPlus is a framework for rigorously evaluating large language models on code related tasks.github.com · read 11 Oct 2026
- RepoQA
- The EvalPlus site describes RepoQA as work on evaluators for long-context code understanding.evalplus.github.io · read 11 Oct 2026
- Security
- The README describes Docker-based code execution as safe code execution.github.com · read 5 Oct 2026
- Use by model teams
- The README says EvalPlus datasets have been used by teams including Meta Llama, Allen AI TÜLU, Qwen, DeepSeek, Snowflake, StarCoder2, Magicoder, and WizardCoder.github.com · read 11 Oct 2026
