Free public ranking

SaaS Robots.txt Leaderboard

How the web's top SaaS companies & web apps compare on openness to AI crawlers. Graded straight from each site's robots.txt against the major AI bots (GPTBot, ClaudeBot, Google-Extended and more). Use this page to see how your site compares, and to check if your robots.txt is blocking or allowing the AI bots you want to allow or block.

1%

of 211 ranked sites block GPTBot in robots.txt

211

sites graded, last checked Nov 11, 2026

99/100

average openness score (higher = more open to AI)

Share of sites blocking each AI crawler

Click a bot to filter the ranking to sites that block it.

Grade distribution

A = wide open to AI, F = walled off.

A
204 sites
B
4 sites
C
2 sites
D
1 site
F
0 sites
# AI crawlers Region
151
Procore
procore.com
A 100 0/11 Allows all US
152
Proton
proton.me
A 100 0/11 Allows all EU
153
Qualtrics
qualtrics.com
A 100 0/11 Allows all US
154
Ramp
ramp.com
A 100 0/11 Allows all US
155
RD Station
rdstation.com
A 100 0/11 Allows all LatAm
156
Redis
redis.io
A 100 0/11 Allows all US
157
Render
render.com
A 100 0/11 Allows all US
158
Replit
replit.com
A 100 0/11 Allows all US
159
Retool
retool.com
A 100 0/11 Allows all US
160
RingCentral
ringcentral.com
A 100 0/11 Allows all US
161
Rippling
rippling.com
A 100 0/11 Allows all US
162
Runway
runwayml.com
A 100 0/11 Allows all US
163
SafetyCulture
safetyculture.com
A 100 0/11 Allows all Oceania
164
Sage
sage.com
A 100 0/11 Allows all UK
165
Salesforce
salesforce.com
A 100 0/11 Allows all US
166
Semrush
semrush.com
A 100 0/11 Allows all US
167
Sentry
sentry.io
A 100 0/11 Allows all US
168
ServiceNow
servicenow.com
A 100 0/11 Allows all US
169
Shopify
shopify.com
A 100 0/11 Allows all CA
170
Shopware
shopware.com
A 100 0/11 Allows all EU
171
Similarweb
similarweb.com
A 100 0/11 Allows all MENA-Africa
172
Slack
slack.com
A 100 0/11 Allows all US
173
SmartHR
smarthr.jp
A 100 0/11 Allows all Asia
174
Smartsheet
smartsheet.com
A 100 0/11 Allows all US
175
Snowflake
snowflake.com
A 100 0/11 Allows all US
176
Snyk
snyk.io
A 100 0/11 Allows all US
177
Splunk
splunk.com
A 100 0/11 Allows all US
178
Sprinklr
sprinklr.com
A 100 0/11 Allows all US
179
Sprout Social
sproutsocial.com
A 100 0/11 Allows all US
180
Squarespace
squarespace.com
A 100 0/11 Allows all US
181
Stability AI
stability.ai
A 100 0/11 Allows all UK
182
Stripe
stripe.com
A 100 0/11 Allows all US
183
Supabase
supabase.com
A 100 0/11 Allows all US
184
SurveyMonkey
surveymonkey.com
A 100 0/11 Allows all US
185
Synthesia
synthesia.io
A 100 0/11 Allows all UK
186
Tailscale
tailscale.com
A 100 0/11 Allows all CA
187
TeamViewer
teamviewer.com
A 100 0/11 Allows all EU
188
Teamwork.com
teamwork.com
A 100 0/11 Allows all Ireland
189
Tiendanube
tiendanube.com
A 100 0/11 Allows all LatAm
190
Tines
tines.com
A 100 0/11 Allows all Ireland
191
Trello
trello.com
A 100 0/11 Allows all Oceania
192
Twilio
twilio.com
A 100 0/11 Allows all US
193
Typeform
typeform.com
A 100 0/11 Allows all EU
194
Vercel
vercel.com
A 100 0/11 Allows all US
195
Veriff
veriff.com
A 100 0/11 Allows all EU
196
Visma
visma.com
A 100 0/11 Allows all EU
197
Webflow
webflow.com
A 100 0/11 Allows all US
198
Wix
wix.com
A 100 0/11 Allows all MENA-Africa
199
Wiz
wiz.io
A 100 0/11 Allows all US
200
WordPress.com
wordpress.com
A 100 0/11 Allows all US
Showing 151 to 200 of 211 results
Download full leaderboard (CSV)

All 211 graded sites from the latest weekly check, regardless of the filters above.

Methodology

For each site we fetch its public robots.txt and grade it against 11 of the major AI crawlers, including GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot. A site “blocks” a bot when its robots.txt has a Disallow rule that applies to that crawler’s user-agent token. The openness score is the share of graded bots that are allowed, rolled up into an A–F grade.

This measures declared access in robots.txt, not enforcement. robots.txt is a request, not a firewall: a well-behaved crawler obeys it, but it does not technically stop anything. A site can also block AI bots at the edge (WAF, rate limits, IP blocks) in ways this ranking cannot see. Treat the grades as a signal of intent, not a guarantee.

The trend combines two sources: current grades come from a live scan of each site’s robots.txt, re-run every week, while historical snapshots are reconstructed from the Internet Archive (Wayback Machine): we grade the nearest archived robots.txt at or before each quarter. Historical points therefore reflect the best available capture, not necessarily the exact first of the period.

Last checked: Nov 11, 2026, refreshed every week. Reads stored scan data only: no live requests are made when you load this page.

14-day Free Trial

Your Logs already store the data you need,
you just need a way to use it!

If you are interested in HoneyLog, you can try it with our Free Trial!

No credit card required