FOR WEBSITE OWNERS

SEOUnicoBot

How our site-audit crawler behaves

SEOUnicoBot is the crawler used by SEOUnico’s site audit. A person who uses SEOUnico can ask it to audit a website; it then loads that site’s pages to look for technical and on-page SEO issues. It does not index your site for any search engine and it does not store pages beyond the audit results shown to that user.

How it behaves

  • It identifies itself with the user agent SEOUnicoBot/1.0 (+https://seounico.com/bot).
  • It reads and obeys your robots.txt, including Crawl-delay (up to 5 seconds).
  • It requests about one page per second, one at a time, and stops after the page limit of the user’s plan.
  • It stays on the single website being audited and never follows links to other sites.
  • It only fetches HTML pages; images, scripts and downloads are not fetched.

Blocking or limiting it

To block SEOUnicoBot, add this to your robots.txt:

User-agent: SEOUnicoBot
Disallow: /

To slow it down, add Crawl-delay: 5 to its group. You can also block its requests by user agent at your server or firewall.

Questions or problems

If SEOUnicoBot is causing a problem for your website, use our contact page and include your domain.