---
title: "1.5 Million Websites Block at Least One AI Crawler From Reading Their Pages - Suff Digital"
url: https://www.suffdigital.com/resources/data-studies/ai-crawler-blocking-rate
description: "We turn websites into predictable revenue engines. Web design and SEO that drives revenue, not clicks. Nashville-based, senior US team."
lang: en
---

1.5 Million Websites Block at Least One AI Crawler From Reading Their Pages (https://www.suffdigital.com/resources/data-studies/ai-crawler-blocking-rate)

1.5 million websites, 9.9% of the 15.5 million in the public HTTP Archive crawl, block at least one AI crawler. They block the bots that collect training material about four times as often as the bots that fetch pages for live AI answers.

Share of all websites blocking it by AI crawler

GPTBot

8.7%

ClaudeBot / anthropic-ai / Claude-Web

8.3%

CCBot

8.2%

Bytespider

7.9%

Amazonbot

7.8%

Google-Extended

7.6%

Applebot-Extended

7.2%

Meta-ExternalAgent

7.1%

OAI-SearchBot or ChatGPT-User

2.1%

PerplexityBot

1.7%

Source: Suff Digital analysis of 15,470,478 websites · the public HTTP Archive crawl, September 2026

Share of all websites by AI crawler

GPTBot

8.7%

ClaudeBot / anthropic-ai / Claude-Web

8.3%

CCBot

8.2%

Bytespider

7.9%

Amazonbot

7.8%

Google-Extended

7.6%

Applebot-Extended

7.2%

Meta-ExternalAgent

7.1%

OAI-SearchBot or ChatGPT-User

2.1%

PerplexityBot

1.7%

Any AI bot

9.9%

Source: 1.5 Million Websites Block at Least One AI Crawler From Reading Their Pages - Suff Digital (https://www.suffdigital.com/resources/data-studies/ai-crawler-blocking-rate)

## Structured data

```json
{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Organization",
      "@id": "https://www.suffdigital.com/#organization",
      "name": "Suff Digital",
      "alternateName": "Suff Digital Marketing",
      "url": "https://www.suffdigital.com/",
      "logo": {
        "@type": "ImageObject",
        "@id": "https://www.suffdigital.com/#logo",
        "url": "https://www.suffdigital.com/logo.png",
        "contentUrl": "https://www.suffdigital.com/logo.png",
        "caption": "Suff Digital"
      },
      "image": {
        "@id": "https://www.suffdigital.com/#logo"
      },
      "description": "Suff Digital is a full-stack digital marketing agency that turns websites into predictable revenue engines, SEO, paid media, content, CRO, and web design built around revenue, not vanity metrics.",
      "slogan": "Digital marketing built for what's next.",
      "email": "hello@suffdigital.com",
      "telephone": "+1-629-236-0099",
      "foundingDate": "2022",
      "founder": {
        "@type": "Person",
        "name": "Matt Suffoletto",
        "jobTitle": "Founder & CEO"
      },
      "contactPoint": [
        {
          "@type": "ContactPoint",
          "telephone": "+1-629-236-0099",
          "email": "hello@suffdigital.com",
          "contactType": "sales",
          "areaServed": "US",
          "availableLanguage": [
            "en"
          ]
        }
      ],
      "address": {
        "@type": "PostalAddress",
        "addressLocality": "Nashville",
        "addressRegion": "TN",
        "postalCode": "37219",
        "addressCountry": "US"
      },
      "areaServed": {
        "@type": "Country",
        "name": "United States"
      },
      "sameAs": [
        "https://www.linkedin.com/company/suff-digital/",
        "https://www.instagram.com/suffdigital/",
        "https://www.youtube.com/@suffdigitalmarketing",
        "https://www.tiktok.com/@suffdigital"
      ],
      "knowsAbout": [
        "Search Engine Optimization",
        "Pay-Per-Click Advertising",
        "Paid Social Advertising",
        "Conversion Rate Optimization",
        "Web Design",
        "Digital PR",
        "Link Building",
        "Website Migration",
        "Website Maintenance and Support",
        "Technical SEO",
        "Local SEO",
        "AI Search Optimization"
      ],
      "hasOfferCatalog": {
        "@type": "OfferCatalog",
        "name": "Suff Digital Services",
        "itemListElement": [
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Search Engine Optimization (SEO)",
              "url": "https://www.suffdigital.com/"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Pay-Per-Click (PPC) Advertising",
              "url": "https://www.suffdigital.com/services/ppc"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Paid Social Advertising",
              "url": "https://www.suffdigital.com/services/paid-social"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Conversion Rate Optimization (CRO)",
              "url": "https://www.suffdigital.com/services/cro"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Web Design",
              "url": "https://www.suffdigital.com/services/web-design"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Digital PR & Link Building",
              "url": "https://www.suffdigital.com/services/digital-pr"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Website Migration",
              "url": "https://www.suffdigital.com/services/website-migration"
            }
          },
          {
            "@type": "Offer",
            "itemOffered": {
              "@type": "Service",
              "name": "Website Maintenance & Support",
              "url": "https://www.suffdigital.com/services/website-maintenance"
            }
          }
        ]
      }
    },
    {
      "@type": "WebSite",
      "@id": "https://www.suffdigital.com/#website",
      "url": "https://www.suffdigital.com/",
      "name": "Suff Digital",
      "inLanguage": "en-US",
      "publisher": {
        "@id": "https://www.suffdigital.com/#organization"
      }
    }
  ]
}
```