KautharBot
KautharBot: what it does and how to stop it
KautharBot is the crawler used by Kauthar's "SEO & AI visibility" service to audit websites their owners connected. It never crawls a site whose owner did not ask for it.
How it identifies itself
Every request carries the following User-Agent and a link to this page:
Mozilla/5.0 (compatible; KautharBot/1.0; +https://kautharai.com/en/bot)The rules it follows
- It only audits sites their owners connected to Kauthar and proved they own; before that proof it never goes beyond the home page.
- It reads robots.txt and honours Disallow and Crawl-delay (the KautharBot group, then "*"); a disallowed page is never requested.
- One request at a time per site, at least half a second apart, and it backs off on 429 or 503.
- It re-requests with If-None-Match / If-Modified-Since, so unchanged pages are not downloaded again.
- A page cap per site set by the owner's plan, and no single page above 2 MB.
- It collects no personal data, fills no forms and runs no JavaScript.
How to block it
Add these two lines to the robots.txt at the root of your site and KautharBot stops requesting any page:
User-agent: KautharBot
Disallow: /Or block one path and slow it down:
User-agent: KautharBot
Disallow: /private/
Crawl-delay: 2robots.txt is read before every audit, so the change applies from the next audit.
Contact
If you see requests that do not match these rules, write to us with the domain and the time of the request: [email protected]
Let Kauthar answer for you, starting today
Sign up with your phone number, upload your files and share the link with your customers. Kauthar handles the rest, around the clock.
Explore the dashboard with a ready demo account
14-day free trialNo credit cardLive in minutes