- Home
- LLM Evaluation Tools
- WebArena
On the recordRead 11 Oct 202626 facts
WebArena
On the record
- Demo environments
- The demos include social forum, online shopping, content management, collaborative software development, map, wiki, calculator, and scratchpad sites.webarena.dev · read 5 Oct 2026
- Embedded resources
- The environment includes tools and knowledge resources as independent websites.webarena.dev · read 5 Oct 2026
- Environment
- It recreates websites from four popular categories with functionality and data modeled on real-world equivalents, and includes tools and knowledge resources as separate websites.webarena.dev · read 11 Oct 2026
- Evaluation
- Evaluation checks information-seeking answers against annotated references and can programmatically check intermediate execution states.webarena.dev · read 11 Oct 2026
- Evaluation limit
- The repository instructs users to reset the environment after evaluating its 812 examples.github.com · read 11 Oct 2026
- Evaluation requirement
- The repository says to set up your own WebArena websites for correct evaluation and describes the demo sites as browsing-only.github.com · read 5 Oct 2026
- Example tasks
- Shown tasks include creating a repository and checking the status and arrival time of an order.webarena.dev · read 5 Oct 2026
- External model key
- The repository’s evaluation instructions require exporting a valid OpenAI API key before launching evaluation.github.com · read 5 Oct 2026
- Included sites
- The demos include a social forum, online shopping, content management, collaborative software development, a map, a wiki, a calculator, and a scratchpad.webarena.dev · read 11 Oct 2026
- License
- The WebArena code repository identifies its license as Apache-2.0.github.com · read 11 Oct 2026
- Licensing
- The WebArena website states that it is licensed under Creative Commons Attribution-ShareAlike 4.0 International.webarena.dev · read 5 Oct 2026
- Model use
- The repository's example evaluation command uses GPT-3.5 Turbo and requires an OpenAI API key.github.com · read 11 Oct 2026
- Observation formats
- Agents can observe a page through its URL and content, represented as a screenshot, HTML DOM tree, or accessibility tree.webarena.dev · read 5 Oct 2026
- Observation options
- Observations include a page URL and content, represented as a screenshot, HTML DOM tree, or accessibility tree.webarena.dev · read 11 Oct 2026
- Purpose
- WebArena is a standalone, self-hostable web environment and benchmark for building autonomous agents.webarena.dev · read 11 Oct 2026
- Related framework
- The repository describes AgentLab as supporting parallel experiments through BrowserGym, benchmark integration including VisualWebArena, unified leaderboard reporting, and improved handling of environment edge cases.github.com · read 11 Oct 2026
- Research affiliations
- The page lists Carnegie Mellon University and Inspired Cognition as author affiliations.webarena.dev · read 5 Oct 2026
- Research provenance
- The project page lists Carnegie Mellon University and Inspired Cognition affiliations for the paper's authors.webarena.dev · read 11 Oct 2026
