OpenCompass
No phone app
5.5No. 25 of 29
in AI LLM Evaluation Tools
in AI LLM Evaluation Tools
- Recognised40% of the score31
- Phone app26% of the score0
- Documented20% of the score46
- Free plan14% of the score30
- Free plan
- No
- Runs on
- api, Linux, self-hosted
Summary
OpenCompass is ranked #25 of 29 in AI LLM evaluation tools on Samsung Mobile US Press. It runs on API, Linux, Self-hosted.
Compared on AI LLM evaluation tools
- Evaluation methods
- objective; subjective; discriminative; generative; LLM-as-a-judgeopencompass.org.cn
- Model support
- Hugging Face models; API-based models; custom models; OpenAI; Anthropic; Gemini; Qwen; GLM; DeepSeekopencompass.org.cn
- Safety evaluations
- Yesopencompass.org.cn
- Deployment
- self-hostedopencompass.org.cn
Facts
- Purpose
- OpenCompass is a platform for evaluating large language and multimodal models.opencompass.readthedocs.io · 4 Oct 2026
- Benchmarks
- The project describes support for more than 100 datasets covering areas such as knowledge, reasoning, coding, science, language, long context, and safety.github.com · 4 Oct 2026
- Scale
- The project describes distributed evaluation and a scheme with 70+ datasets and about 400,000 questions.github.com · 4 Oct 2026
- Model integrations
- Documented model integrations include OpenAI, Gemini, Claude, Qwen, and other API models, as well as open-source models.github.com · 4 Oct 2026
- API support
- The installation guide documents API evaluation and lists dependencies for OpenAI, Anthropic, Qwen, and ByteDance Volcano Engine APIs.opencompass.readthedocs.io · 4 Oct 2026
- Inference backends
- The installation guide documents LMDeploy and vLLM as inference backend options.opencompass.readthedocs.io · 4 Oct 2026
- Workflow and outputs
- The quick start describes configuring, running inference and evaluation, and saving results as CSV and TXT files.opencompass.readthedocs.io · 4 Oct 2026
- Reporting
- The quick start says users can enable Lark reporting for status updates in Lark clients.opencompass.readthedocs.io · 4 Oct 2026
- Extensibility
- The project says users can add models and datasets and customize task division strategies and cluster management support.github.com · 4 Oct 2026
- License
- The GitHub repository links an Apache-2.0 license.github.com · 4 Oct 2026
- Requirements
- The installation guide states that OpenCompass requires PyTorch 1.13 or later.opencompass.readthedocs.io · 4 Oct 2026
- Dataset limitation
- The basic installation supports most fundamental datasets, while some datasets require additional dependencies or manual setup.opencompass.readthedocs.io · 4 Oct 2026
- Intended users
- The project describes CompassHub and CompassKit for researchers and practitioners evaluating language and vision-language models.github.com · 4 Oct 2026
Best OpenCompass alternatives
See all 12
Where it ranks on Samsung Mobile US Press
- Best AI LLM Evaluation Tools in 2026#25 of 29
Is OpenCompass yours?
Claim it for free: prove the domain, then correct facts, plans and screenshots. An editor reviews every change.
Sources
- opencompass.readthedocs.io/en/doc_refactor/index.html· checked 4 Oct 2026
- github.com/open-compass/opencompass· checked 4 Oct 2026
- opencompass.readthedocs.io/en/latest/get_started/installation.html· checked 4 Oct 2026
- opencompass.readthedocs.io/en/latest/get_started/quick_start.html· checked 4 Oct 2026