Free public ranking

SaaS Robots.txt Leaderboard

How the web's top SaaS companies & web apps compare on openness to AI crawlers. Graded straight from each site's robots.txt against the major AI bots (GPTBot, ClaudeBot, Google-Extended and more). Use this page to see how your site compares, and to check if your robots.txt is blocking or allowing the AI bots you want to allow or block.

1%

of 211 ranked sites block GPTBot in robots.txt

211

sites graded, last checked Nov 11, 2026

99/100

average openness score (higher = more open to AI)

Share of sites blocking each AI crawler

Click a bot to filter the ranking to sites that block it.

Grade distribution

A = wide open to AI, F = walled off.

A
204 sites
B
4 sites
C
2 sites
D
1 site
F
0 sites
# AI crawlers Region
201
Workday
workday.com
A 100 0/11 Allows all US
202
Workhuman
workhuman.com
A 100 0/11 Allows all Ireland
203
WPS Office
wps.com
A 100 0/11 Allows all Asia
204
Wrike
wrike.com
A 100 0/11 Allows all US
205
Xero
xero.com
A 100 0/11 Allows all Oceania
206
Zapier
zapier.com
A 100 0/11 Allows all US
207
Zendesk
zendesk.com
A 100 0/11 Allows all US
208
Zenvia
zenvia.com
A 100 0/11 Allows all LatAm
209
Zoho
zoho.com
A 100 0/11 Allows all Asia
210
Zoom
zoom.us
A 100 0/11 Allows all US
211
Zscaler
zscaler.com
A 100 0/11 Allows all US
Showing 201 to 211 of 211 results
Download full leaderboard (CSV)

All 211 graded sites from the latest weekly check, regardless of the filters above.

Methodology

For each site we fetch its public robots.txt and grade it against 11 of the major AI crawlers, including GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot. A site “blocks” a bot when its robots.txt has a Disallow rule that applies to that crawler’s user-agent token. The openness score is the share of graded bots that are allowed, rolled up into an A–F grade.

This measures declared access in robots.txt, not enforcement. robots.txt is a request, not a firewall: a well-behaved crawler obeys it, but it does not technically stop anything. A site can also block AI bots at the edge (WAF, rate limits, IP blocks) in ways this ranking cannot see. Treat the grades as a signal of intent, not a guarantee.

The trend combines two sources: current grades come from a live scan of each site’s robots.txt, re-run every week, while historical snapshots are reconstructed from the Internet Archive (Wayback Machine): we grade the nearest archived robots.txt at or before each quarter. Historical points therefore reflect the best available capture, not necessarily the exact first of the period.

Last checked: Nov 11, 2026, refreshed every week. Reads stored scan data only: no live requests are made when you load this page.

14-day Free Trial

Your Logs already store the data you need,
you just need a way to use it!

If you are interested in HoneyLog, you can try it with our Free Trial!

No credit card required