- Home
- AI LLM Evaluation Tools
- OpenCompass
On the recordRead 8 Oct 202623 facts
OpenCompass
On the record
- API support
- The installation guide documents API evaluation and lists dependencies for OpenAI, Anthropic, Qwen, and ByteDance Volcano Engine APIs.opencompass.readthedocs.io · read 4 Oct 2026
- Benchmarks
- The project describes support for more than 100 datasets covering areas such as knowledge, reasoning, coding, science, language, long context, and safety.github.com · read 4 Oct 2026
- Community support
- The README links to Discord and WeChat and directs users to GitHub for reporting issues.github.com · read 8 Oct 2026
- Compatibility limit
- The README says APPS, TACO, and LiveCodeBench code-generation evaluations are unavailable without pyext, and recommends Python 3.10 for those datasets.github.com · read 8 Oct 2026
- Dataset limitation
- The basic installation supports most fundamental datasets, while some datasets require additional dependencies or manual setup.opencompass.readthedocs.io · read 4 Oct 2026
- Datasets
- OpenCompass publishes recommended configurations for supported evaluation datasets.doc.opencompass.org.cn · read 8 Oct 2026
- Distributed evaluation
- The project says it supports task division and distributed evaluation.github.com · read 8 Oct 2026
- Evaluation dimensions
- Its documentation describes evaluations across language, knowledge, understanding, reasoning, safety, and specialized capabilities.doc.opencompass.org.cn · read 8 Oct 2026
- Evaluation methods
- It supports objective and subjective evaluation, including model-assisted judging and human-feedback-based judging.doc.opencompass.org.cn · read 8 Oct 2026
- Extensibility
- The project says users can add models and datasets and customize task division strategies and cluster management support.github.com · read 4 Oct 2026
- Inference backends
- The README gives LMDeploy and vLLM as accelerated inference backend options.github.com · read 8 Oct 2026
- Installation
- The project documents installation with pip and from source, and its README recommends using conda to manage the Python environment.github.com · read 8 Oct 2026
- Intended users
- The project describes CompassHub and CompassKit for researchers and practitioners evaluating language and vision-language models.github.com · read 4 Oct 2026
- License
- The GitHub repository links an Apache-2.0 license.github.com · read 4 Oct 2026
- Model integrations
- Documented model integrations include OpenAI, Gemini, Claude, Qwen, and other API models, as well as open-source models.github.com · read 4 Oct 2026
- Model support
- The README describes support for Hugging Face models, API models, and custom models.github.com · read 8 Oct 2026
- Purpose
- OpenCompass is a platform and toolkit for evaluating large language models and large vision-language models.github.com · read 8 Oct 2026
- Reporting
- The quick start says users can enable Lark reporting for status updates in Lark clients.opencompass.readthedocs.io · read 4 Oct 2026
