Free public ranking

SaaS Robots.txt Leaderboard

How the web's top SaaS companies & web apps compare on openness to AI crawlers. Graded straight from each site's robots.txt against the major AI bots (GPTBot, ClaudeBot, Google-Extended and more). Use this page to see how your site compares, and to check if your robots.txt is blocking or allowing the AI bots you want to allow or block.

1%

of 211 ranked sites block GPTBot in robots.txt

211

sites graded, last checked Nov 11, 2026

99/100

average openness score (higher = more open to AI)

Share of sites blocking each AI crawler

Click a bot to filter the ranking to sites that block it.

Grade distribution

A = wide open to AI, F = walled off.

A
204 sites
B
4 sites
C
2 sites
D
1 site
F
0 sites
# AI crawlers Region
1
Canva
canva.com
D 45 6/11
GPTBot ClaudeBot CCBot Bytespider Applebot-Extended Meta-ExternalAgent
Oceania
2
Figma
figma.com
C 55 5/11
GPTBot ClaudeBot Google-Extended CCBot Bytespider
US
3
ZoomInfo
zoominfo.com
C 64 4/11
CCBot Bytespider Amazonbot Meta-ExternalAgent
US
4
Attio
attio.com
B 82 2/11
CCBot Bytespider
UK
5
Descript
descript.com
B 82 2/11
Bytespider Amazonbot
US
6
Kajabi
kajabi.com
B 82 2/11
CCBot Bytespider
US
7
Notion
notion.com
B 82 2/11
Amazonbot Meta-ExternalAgent
US
8
ActiveCampaign
activecampaign.com
A 91 1/11
Amazonbot
US
9
Amplitude
amplitude.com
A 91 1/11
Bytespider
US
10
Calendly
calendly.com
A 91 1/11
CCBot
US
11
FreshBooks
freshbooks.com
A 91 1/11
Bytespider
CA
12
GitHub
github.com
A 91 1/11
Bytespider
US
13
Hostinger
hostinger.com
A 91 1/11
Bytespider
EU
14
Postman
postman.com
A 91 1/11
Bytespider
US
15
Razorpay
razorpay.com
A 91 1/11
Bytespider
Asia
16
1Password
1password.com
A 100 0/11 Allows all CA
17
Adobe
adobe.com
A 100 0/11 Allows all US
18
Adyen
adyen.com
A 100 0/11 Allows all EU
19
Ahrefs
ahrefs.com
A 100 0/11 Allows all Asia
20
AI21 Labs
ai21.com
A 100 0/11 Allows all MENA-Africa
21
Aircall
aircall.io
A 100 0/11 Allows all EU
22
Airtable
airtable.com
A 100 0/11 Allows all US
23
Algolia
algolia.com
A 100 0/11 Allows all US
24
Anthropic
anthropic.com
A 100 0/11 Allows all US
25
Apollo.io
apollo.io
A 100 0/11 Allows all US
26
Asana
asana.com
A 100 0/11 Allows all US
27
Atlassian
atlassian.com
A 100 0/11 Allows all Oceania
28
Auth0
auth0.com
A 100 0/11 Allows all US
29
Autodesk
autodesk.com
A 100 0/11 Allows all US
30
BambooHR
bamboohr.com
A 100 0/11 Allows all US
31
Basecamp
basecamp.com
A 100 0/11 Allows all US
32
BigCommerce
bigcommerce.com
A 100 0/11 Allows all US
33
BILL
bill.com
A 100 0/11 Allows all US
34
Bitwarden
bitwarden.com
A 100 0/11 Allows all US
35
Box
box.com
A 100 0/11 Allows all US
36
Brandwatch
brandwatch.com
A 100 0/11 Allows all UK
37
Braze
braze.com
A 100 0/11 Allows all US
38
Brevo
brevo.com
A 100 0/11 Allows all EU
39
Brex
brex.com
A 100 0/11 Allows all US
40
Bubble
bubble.io
A 100 0/11 Allows all US
41
Buffer
buffer.com
A 100 0/11 Allows all US
42
Celonis
celonis.com
A 100 0/11 Allows all EU
43
Channel Talk
channel.io
A 100 0/11 Allows all Asia
44
Checkout.com
checkout.com
A 100 0/11 Allows all UK
45
CircleCI
circleci.com
A 100 0/11 Allows all US
46
ClickUp
clickup.com
A 100 0/11 Allows all US
47
Clio
clio.com
A 100 0/11 Allows all CA
48
Cloudflare
cloudflare.com
A 100 0/11 Allows all US
49
Cohere
cohere.com
A 100 0/11 Allows all CA
50
Confluent
confluent.io
A 100 0/11 Allows all US
Showing 1 to 50 of 211 results
Download full leaderboard (CSV)

All 211 graded sites from the latest weekly check, regardless of the filters above.

Methodology

For each site we fetch its public robots.txt and grade it against 11 of the major AI crawlers, including GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot. A site “blocks” a bot when its robots.txt has a Disallow rule that applies to that crawler’s user-agent token. The openness score is the share of graded bots that are allowed, rolled up into an A–F grade.

This measures declared access in robots.txt, not enforcement. robots.txt is a request, not a firewall: a well-behaved crawler obeys it, but it does not technically stop anything. A site can also block AI bots at the edge (WAF, rate limits, IP blocks) in ways this ranking cannot see. Treat the grades as a signal of intent, not a guarantee.

The trend combines two sources: current grades come from a live scan of each site’s robots.txt, re-run every week, while historical snapshots are reconstructed from the Internet Archive (Wayback Machine): we grade the nearest archived robots.txt at or before each quarter. Historical points therefore reflect the best available capture, not necessarily the exact first of the period.

Last checked: Nov 11, 2026, refreshed every week. Reads stored scan data only: no live requests are made when you load this page.

14-day Free Trial

Your Logs already store the data you need,
you just need a way to use it!

If you are interested in HoneyLog, you can try it with our Free Trial!

No credit card required