EvaristBot

Every request Evarist sends to a customer's site carries this exact User-Agent, verbatim:

Mozilla/5.0 (compatible; EvaristBot/1.0; +https://evarist.ai/bot)

The product token is EvaristBot — match it in a robots.txt group, a WAF or CDN bot rule, or your access logs. The version number identifies crawler behaviour, not a release; it changes only when what we do changes.

We read robots.txt first and obey it before fetching anything. Your homepage renders next — in a real browser, so client-rendered sites read correctly, not just their initial HTML — then we read sitemap.xml to plan the rest of the crawl.

Three other kinds of request carry the same User-Agent but aren't the crawl: an account holder authoring or checking a no-code tag, an account holder running an install check, and our tag-health monitor automatically re-checking a tag that looks broken. Each re-fetches a single page already known from your site's model — never a newly discovered one — so robots.txt, covered next, governs the crawl and doesn't gate these.

• Before crawling, we read robots.txt and obey it, including a group addressed specifically to EvaristBot. This governs the crawl — the owner-triggered, one-off fetches described above aren't autonomous crawling, so this rule doesn't apply to them.
• Normally only one crawl runs against a site at a time.
• Concurrent requests stay low — typically single digits, low double digits at most during a brief overlap between a scheduled crawl and a refresh.
• A first crawl covers up to a couple hundred pages; later refreshes are much smaller.
• We refresh roughly every couple of weeks, and stop entirely for sites that go quiet.

Match the EvaristBot token in your WAF's or CDN's bot rules — Cloudflare, Akamai and similar platforms let you allow (or block) a request by its user-agent string. Some platforms — Akamai's custom bot categories, for instance — also let you define your own named bot category this way. A pre-built "verified bot" directory, where a platform has one, is typically populated by the bot operator applying to be listed, not something a site owner adds a bot to from their own dashboard — so it isn't where you'll find EvaristBot; the custom rule above is the one to use.

A user-agent identifies, it doesn't authenticate. Anyone can send this header, so treat a UA rule as "let this crawler through", never as proof of who is calling. Scope it to what you'd let any well-behaved crawler do — reading pages you already publish — and don't let it bypass authentication or reach anything you wouldn't serve to the open web. If you need a rule that can't be spoofed, email support@evarist.ai and we'll give you our current egress addresses to pair it with.

We honour a robots.txt rule that disallows us:

User-agent: EvaristBot
Disallow: /

A group naming EvaristBot replaces the * group for us entirely — it doesn't add to it. Scope the rules under your EvaristBot group completely; we won't fall back to whatever the * group also allows or disallows.