CrawlerCheck

Automated

Free online tool to verify and diagnose web crawlers and bots accessing your site, identify legitimate search engine crawlers, and block unwanted scrapers.

Visit website Pricing page

Active plans

Total changes

Last change

—

Price trend

no history

Current pricing

Free

monthly

Free

Multi-crawler permission analysis
robots.txt validation
Meta robots tag checking
X-Robots-Tag HTTP header analysis
Real-time crawlability testing

Price history

Last 0 changes

No price changes recorded yet.

Similar to CrawlerCheck.

No confirmed competitors yet. These are Data Engineering products at a similar price, matched automatically — not verified alternatives.

Agenty

From $29/mo

No-code web scraping platform for data extraction with scheduling, proxy support, API access, and team management.

BotIntelli

From $29/mo

AI bot platform with RAG-powered chat, knowledge base, workflow automation, and data pipelines.

Covie

From 12¢/event (Tier 1)

Insurance policy data API with Access (retrieve policies) and Monitor (track changes) services, usage-based pricing.

DataFuel

From $26/mo

Web scraping API that converts websites into LLM-ready datasets with AI JSON schema extraction, automated login, retries, and crawler for RAG and AI applications.

Datagma

From $49/mo

B2B data enrichment platform for finding mobile phones and verified emails with Sales Navigator export support.

Endato

From $0.25/match

Endato is a public records API platform providing access to people search, property, business, assets, and criminal court data via live API calls.