- Home
- SEO Log File Analysis Tools
- Crawler Log Analyser
On the recordRead 10 Oct 202623 facts
Crawler Log Analyser
Inverness, Scottish Highlands, UK
On the record
- Additional detections
- It can detect sitemap-path user-agent spoofing and rotating bot identity claims from a single IP.github.com · read 4 Oct 2026
- Anomaly detection
- It detects crawler-visible deploy-window issues including robots.txt failures, content-404 bursts, and post-404 crawl gaps.github.com · read 10 Oct 2026
- Configuration
- A TOML config file can specify ignored source IPs, site launch dates, and an AI crawler discovery window.github.com · read 10 Oct 2026
- Crawler analysis
- It classifies search and AI crawlers and scores Googlebot and broader search-crawler health from observed responses.github.com · read 4 Oct 2026
- Crawler insights
- It classifies search and AI crawlers and scores Googlebot crawl health from observed responses.github.com · read 10 Oct 2026
- Data handling
- The README says the tool has no database, telemetry, or external services.github.com · read 10 Oct 2026
- Deploy anomalies
- It detects crawler-visible deploy-window issues such as robots.txt failures, content-404 bursts, and post-404 crawl gaps.github.com · read 4 Oct 2026
- Feature status
- The feature-freeze page says no new feature work was planned through 31 August 2026 and that bug fixes and security fixes remain welcome.github.com · read 10 Oct 2026
- Integrations
- The README says JSON output is the integration boundary and lists no Slack, Prometheus, Elasticsearch, or SaaS integrations.github.com · read 10 Oct 2026
- Intended users
- The maker positions it for operators of self-hosted static sites who need to inspect real crawler behavior in nginx access logs.github.com · read 4 Oct 2026
- License
- The project is licensed under AGPL-3.0-or-later; network services based on modified versions must make corresponding source code available under the AGPL.github.com · read 10 Oct 2026
- Log analysis
- It parses nginx access logs, including rotated .gz logs, and supports combined and extended seo_crawl formats.github.com · read 10 Oct 2026
- Log parsing
- It parses nginx access logs in combined or extended seo_crawl format, including rotated .gz logs.github.com · read 4 Oct 2026
- Maker
- SpeyTech identifies Spey Systems Ltd as the company behind its repositories and describes it as a Scottish software and systems engineering company based in Inverness.github.com · read 10 Oct 2026
- Privacy and architecture
- The maker describes it as a single-file Python script with no telemetry, dashboards, database, external services, or external dependencies.speytech.com · read 4 Oct 2026
- Project status
- The repository identifies v1.10.2 as its status and says feature work was frozen through 31 August 2026, while bug fixes and security fixes remain welcome.github.com · read 4 Oct 2026
- Purpose
- The project describes itself as operational SEO observability for self-hosted static sites running nginx.github.com · read 10 Oct 2026
- Reports
- It emits text, Markdown, or JSON reports and can exit non-zero on critical findings for cron and CI use.github.com · read 4 Oct 2026
