WhiteTieCheck crawler

When someone runs a check on a website, we make a small number of requests to that website to read publicly available information. Requests identify themselves as:

WhiteTieCheck/1.0 (+https://whitetie.ai/bot)

What it fetches

• Only the website submitted for the check: the homepage, an About page, robots.txt, and llms.txt. If the homepage turns it away, it also reads the website's sitemap (and the first sitemap it lists) to see what it can still reach.
• It never fetches competitors' websites, never runs JavaScript, never pretends to be a browser or another crawler, and never tries to get around a block, challenge, or rate limit.
• It respects robots.txt rules for WhiteTieCheck.

Blocking it

Add this to your robots.txt:

User-agent: WhiteTieCheck
Disallow: /

If your website turns our request away, the check says so. That alone doesn't mean AI assistants are blocked. The report goes by your robots.txt rules for each AI crawler instead, and says so plainly when it can't read them.

Questions: hello@whitetieseo.com.