- Home
- Website Audit software
- SiteOne Crawler
SiteOne Crawler
What it is
SiteOne Crawler is a website crawling and analysis tool for reviewing SEO, performance, accessibility, security, best practices, and content. It produces interactive HTML reports as well as text and JSON output, and assigns scores from 0.0 to 10.0 across five quality areas: performance, SEO, security, accessibility, and best practices. It can export a static website copy for offline browsing, convert a whole site to Markdown, and create llms.txt. For JavaScript-driven pages, an optional Chromium mode renders the pages before the crawl. The tool offers desktop and command-line variants for Windows, macOS, and Linux, as well as API access and self-hosting. Its Rust binary has no runtime dependencies, so Node, Python, and Docker are not required. CI/CD checks can use configurable thresholds, JUnit XML, GitHub annotations, and exit code 10 on failure. Reports can be shared using secure URLs with optional expiry and password protection. The default crawl uses three workers and no more than 10 requests per second combined. SiteOne Crawler is free and open source, with no account required.
For and against
For
- Creates HTML, text, and JSON reports.
- Exports static copies and Markdown with llms.txt.
- Optional Chromium mode handles JavaScript-rendered pages.
- CI/CD gate supports thresholds and GitHub annotations.
- Runs on Windows, macOS, and Linux without runtime dependencies.
Against
- Default crawl rate is capped at 10 requests per second combined.
- Optional browser mode is needed for rendered JavaScript pages.
The verdict
SiteOne Crawler combines site auditing, export options, and CI/CD checks in a free tool. Its multiple output formats and operating-system support make it suitable for varied crawling workflows.
Who it suits
It suits website owners, developers, QA engineers, administrators, DevOps practitioners, and consultants who need to crawl or assess sites. Its API and CI/CD options may also fit teams integrating checks into their workflows.
Plans and prices
Free
Open-source website crawling and analysis · CLI and desktop application · HTML, text, and JSON reports
Read 21 Sep 2026 · crawler.siteone.io
SiteOne Crawler
Free, open-source alternative; no account required
Read 7 Oct 2026 · crawler.siteone.io
On the record
- AI integrations
- Its opt-in AI assistant supports OpenAI, Anthropic, Google Gemini, vLLM, Ollama, LM Studio, and OpenAI-compatible endpoints.crawler.siteone.io · read 7 Oct 2026
- API access
- Yescrawler.siteone.io · read 21 Sep 2026
- Audience
- The maker identifies website owners, developers, QA engineers, administrators, DevOps practitioners, and consultants as intended users.crawler.siteone.io · read 7 Oct 2026
- CI/CD
- The built-in CI/CD quality gate supports configurable thresholds, JUnit XML, GitHub annotations, and exit code 10 on failure.crawler.siteone.io · read 7 Oct 2026
- Default crawl rate
- By default, it uses three workers and no more than 10 requests per second combined.crawler.siteone.io · read 7 Oct 2026
- JavaScript sites
- Optional browser mode uses Chromium to render pages so JavaScript-driven sites can be crawled with their post-render DOM.crawler.siteone.io · read 7 Oct 2026
- Markdown export
- It can convert an entire website to Markdown and generate llms.txt.crawler.siteone.io · read 7 Oct 2026
- Offline copy
- It can export a static copy of a website for offline browsing.crawler.siteone.io · read 7 Oct 2026
- Purpose
- SiteOne Crawler crawls websites and analyzes SEO, performance, accessibility, security, best practices, and content, then produces an interactive HTML report or text and JSON output.crawler.siteone.io · read 7 Oct 2026
- Quality scoring
- It assigns a 0.0–10.0 quality score across performance, SEO, security, accessibility, and best practices.crawler.siteone.io · read 7 Oct 2026
- Report sharing
- A free report-sharing service can provide a secure URL with optional expiration and password protection.crawler.siteone.io · read 7 Oct 2026
- Runtime dependencies
- The Rust binary has zero runtime dependencies, with no Node, Python, or Docker required.crawler.siteone.io · read 7 Oct 2026
- Site audit
- Yescrawler.siteone.io · read 21 Sep 2026
- Support
- The maker lists GitHub Issues for bugs and feature requests and GitHub Discussions for questions and discussion.crawler.siteone.io · read 7 Oct 2026
- Supported systems
- The single native binary runs on Windows, macOS, and Linux, with desktop and command-line variants.crawler.siteone.io · read 7 Oct 2026
