A

Bot policy

AI Commerce Audit runs a rate-limited crawler against any store URL submitted by a merchant. This page describes how to identify our traffic and what to expect.

User-Agent

AiCommerceAuditBot/1.0 (+https://aicommerceaudit.com/bot)

Behavior

  • Single-thread per host. We never run more than one in-flight request against a given hostname at a time.
  • 1 request / second / host, burst 5.
  • Honors robots.txt. If you disallow our UA (AiCommerceAuditBot) at any path, we will not request it. (Note: if you block us, you also cannot audit your own store from our public form.)
  • Up to 5 redirects, 8s timeout, single retry on 5xx.
  • SSRF-safe. We reject redirects to RFC1918, link-local, loopback, and cloud-metadata IPs.
  • Cookie + auth stripping on cross-origin redirects.
  • Sample size: up to 20 product detail pages per audit.

What we fetch

  • / — the homepage
  • /robots.txt
  • /sitemap.xml (+ one child sitemap from a sitemap-index if present)
  • /llms.txt
  • /.well-known/ucp
  • /products.json (Shopify), /wp-json/wc/store/products (WooCommerce), or sitemap-driven product paths
  • Up to 20 product detail pages from the platform sample

We do not authenticate, do not POST, do not run JavaScript, do not download images, and do not honor session cookies.

Allowlisting

Most stores need no action. If you run a strict WAF (Cloudflare bot challenge, Akamai Bot Manager, etc.), allowlisting our UA string above will let monitored audits run cleanly.

Whatever firewall you use, allow requests whose User-Agent contains:

AiCommerceAuditBot/1.0 (+https://aicommerceaudit.com/bot)

Vendor-specific steps follow. Jump to your provider: Vercel, Cloudflare, Akamai, AWS (CloudFront / WAF), Imperva (Incapsula), DataDome, HUMAN (PerimeterX), Sucuri.

Vercel

  1. Open your Vercel project → Settings → Firewall (or Security).
  2. Add a rule that allows requests whose User-Agent contains "AiCommerceAuditBot", and exempt that rule from Bot Protection / Attack Challenge Mode.
  3. Re-run the audit.

Cloudflare

  1. Cloudflare dashboard → your domain → Security → WAF.
  2. Create a custom rule: when User Agent contains "AiCommerceAuditBot", set the action to Skip — and skip Bot Fight Mode and managed challenges.
  3. If you use Super Bot Fight Mode, allow verified/known bots too.
  4. Re-run the audit.

Akamai

  1. In Akamai Bot Manager, open your bot category / allow lists.
  2. Add "AiCommerceAuditBot" as an allowed bot so it isn't challenged.
  3. Re-run the audit.

AWS (CloudFront / WAF)

  1. In AWS WAF, edit the web ACL attached to your CloudFront distribution.
  2. Add an Allow rule for requests whose User-Agent contains "AiCommerceAuditBot", placed above your bot-control / blocking rules.
  3. Re-run the audit.

Imperva (Incapsula)

  1. In the Imperva dashboard, open your site's Bot Access Control settings.
  2. Add "AiCommerceAuditBot" as an allowed client / good bot.
  3. Re-run the audit.

DataDome

  1. In the DataDome dashboard, open Bots → Allowlist (or Custom Rules).
  2. Allow the user-agent "AiCommerceAuditBot".
  3. Re-run the audit.

HUMAN (PerimeterX)

  1. In the HUMAN / PerimeterX console, open your allowlist rules.
  2. Add a rule that allows the user-agent "AiCommerceAuditBot".
  3. Re-run the audit.

Sucuri

  1. In the Sucuri firewall dashboard → Settings → Access Control.
  2. Whitelist the user-agent "AiCommerceAuditBot".
  3. Re-run the audit.

Public leaderboard

Stores audited anonymously from the homepage form join the public leaderboard by default. Own the store and want yours hidden? Sign in, save the store to your dashboard, and verify ownership to hide it from the public leaderboard. We honor the change on the next page render.

Need a store removed entirely? Email hi@aicommerceaudit.com with the store URL — we process takedowns by hand within one business day.

Contact

Found our bot misbehaving? Email hi@aicommerceaudit.com with the request timestamp + path.