DirectoryToolkitBot
If you found this address in your server logs, this page explains what visited your site and how to stop it.
What it is
DirectoryToolkitBot is the website reader in DRXL ONE's Directory Operator Toolkit. Directories and franchisors hire us to tidy up their business listings and write accurate descriptions of the businesses they list. To describe a business accurately, we read that business's own public website, the same pages any visitor can see.
We use what we read to draft a short listing description for the directory that hired us, and to note facts your site publishes about the business: services, areas served and published contact details, each with the page it came from. A person reviews every description before it's published. What we read goes only to the directory that hired us, for its listing of your business; we don't sell it.
How it behaves
- It reads robots.txt first and follows it. Rules for "DirectoryToolkitBot" apply if you have them; otherwise the rules for all crawlers (User-agent: *) apply. If your robots.txt can't be read because your server errors or times out, it leaves your site alone.
- It reads a handful of pages: usually your home page plus a few pages like About or Services, never more than ten. It waits between pages and stops after three failed requests in a row.
- It only follows links within your own site, and skips login, account, cart, checkout and admin pages.
- It reads page text only. It doesn't submit forms, sign in, download files or run your site's scripts.
- It connects on the standard web ports only (80 and 443), and identifies itself on every request.
It identifies itself with a user agent that starts with DirectoryToolkitBot/ and links back to this page.
How to block it
Add this to your site's robots.txt:
User-agent: DirectoryToolkitBot Disallow: /
It takes effect on the next visit. To ask about a visit, or to have your business left out of our work entirely, email carl@drxlone.com.