--- title: "GPTBot Blocking Statistics: 47% of Top News Sites Block It" url: https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot description: "GPTBot blocking statistics: 47% of the top news websites in Google News block OpenAI's training crawler, against 17.6% of all news sites. See the data." lang: en --- Share Share on X: https://twitter.com/intent/tweet?url=https%3A%2F%2Fwww.suffdigital.com%2Fresources%2Fdata-studies%2Fnews-sites-block-gptbot&text=47%25%20of%20Top%20News%20Sites%20Block%20ChatGPT%27s%20Training%20Crawler%20vs%2017.6%25%20of%20All%20News%20Sites Share on Facebook: https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fwww.suffdigital.com%2Fresources%2Fdata-studies%2Fnews-sites-block-gptbot Share on LinkedIn: https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fwww.suffdigital.com%2Fresources%2Fdata-studies%2Fnews-sites-block-gptbot Share on Reddit: https://www.reddit.com/submit?url=https%3A%2F%2Fwww.suffdigital.com%2Fresources%2Fdata-studies%2Fnews-sites-block-gptbot&title=47%25%20of%20Top%20News%20Sites%20Block%20ChatGPT%27s%20Training%20Crawler%20vs%2017.6%25%20of%20All%20News%20Sites **47%** of the news websites Google News shows most often block GPTBot, the crawler OpenAI uses to collect training material for ChatGPT, against 17.6% of all news publisher websites. Far fewer block the bots ChatGPT uses to fetch and cite pages when it answers. Analysis of 178 top Google News websites and 52,326 news publisher websites · Google News results and the public HTTP Archive crawl, September 2026 ## Key Findings 1. 1. **47%** of the top news websites in Google News block OpenAI's GPTBot training crawler, against 17.6% of all news publisher websites. Copy stat 2. 2. **61.2%** of top news websites block at least one AI crawler, and 54.5% block at least one of OpenAI's three bots. Copy stat 3. 3. **21.3%** of top news websites block OAI-SearchBot, the bot behind ChatGPT's search results, less than half the 47% that block GPTBot. Copy stat 4. 4. Among top news websites that also rank in the 10,000 most visited sites on the web, **55.2%** block GPTBot. Copy stat 5. 5. Reuters blocks ChatGPT's search and browsing bots yet is ChatGPT's most-cited news outlet, in **250** news answers: a robots.txt block does not by itself remove an outlet from ChatGPT's answers. Copy stat ## Summary Every website can post a short file, robots.txt, that tells automated visitors what they may read. Since AI companies began collecting the web to train their models, newsrooms have been using that file to say no. The New York Times, USA Today, NBC News, NPR and CNN all turn away every OpenAI bot. That choice has two sides for a publisher. Blocking the training crawler keeps articles out of the material future AI models learn from, which protects the value of the reporting. Blocking the bots that fetch pages for live answers can also keep the outlet out of ChatGPT's citations, and so out of a growing source of readers. For brands and PR teams, it decides which outlets' coverage an AI engine can read and quote. The biggest news brands block far more than the rest. **47%** of the news websites that appear most often in Google News block GPTBot, against 17.6% of all 52,326 news publisher websites. But most of them treat training and search differently: 21.3% block OAI-SearchBot, the bot behind ChatGPT's search results. ## What we measured We took the 200 websites that appeared most often in 2,000 Google News result pages collected on September 23, 2026, and matched 178 of them to their homepage in the September 2026 HTTP Archive crawl, a public record of millions of websites. We read each site's robots.txt file and checked which AI crawlers it blocks. For comparison, we ran the same check on all 52,326 websites that identify themselves as news publishers in their page code. A crawler, or bot, is a program that visits web pages automatically. OpenAI runs three: GPTBot collects pages to train its models, ChatGPT-User fetches a page when a ChatGPT user asks about it, and OAI-SearchBot gathers pages for ChatGPT's search results. A site blocks a bot when its robots.txt file tells that bot to stay off the whole site. ## Nearly half of top news sites block GPTBot 47% of the top news websites in Google News block OpenAI's GPTBot. 84 of the 178 top news websites turn GPTBot away. The share rises to 54.5% for at least one of OpenAI's three bots, and to 61.2% for at least one AI crawler of any kind. 6.7% block other AI companies' crawlers while leaving all of OpenAI's bots alone. Top news websites blocking it by bot Any AI crawler 61.2% Any OpenAI bot 54.5% GPTBot (training) 47.2% ChatGPT-User (live browsing) 42.1% OAI-SearchBot (ChatGPT search) 21.3% Source: Suff Digital analysis of 178 top Google News websites and 52,326 news publisher websites · Google News results and the public HTTP Archive crawl, September 2026 Across all news publisher websites the picture is very different: 17.6% block GPTBot. The top outlets block it at more than two and a half times that rate. Blocking AI is mostly a big-publisher decision. ## News sites block training more than live answers Top news websites are far more willing to block training than search. 47% block GPTBot, 42.1% block ChatGPT-User and 21.3% block OAI-SearchBot. Many publishers keep their articles out of model training while leaving the door open for ChatGPT to find and cite them in search. | Outlet | Blocks GPTBot | Blocks ChatGPT-User | Blocks OAI-SearchBot | | --- | --- | --- | --- | | nytimes.com | Yes | Yes | Yes | | usatoday.com | Yes | Yes | Yes | | nbcnews.com | Yes | Yes | Yes | | npr.org | Yes | Yes | Yes | | cnn.com | Yes | Yes | Yes | | apnews.com | Yes | No | No | | cbsnews.com | Yes | No | No | | reuters.com | No | Yes | Yes | | foxnews.com | No | No | No | | theguardian.com | No | No | No | | washingtonpost.com | No | No | No | The best-known outlets split into clear camps. The New York Times, USA Today, NBC News, NPR, CNN, People and Yahoo block all three OpenAI bots. AP News, CBS News and the Los Angeles Times block GPTBot only, so they stay reachable for ChatGPT's answers. Reuters does the reverse, allowing GPTBot and blocking the other two. Fox News, the Guardian and the Washington Post block none of the three. ## The biggest publishers block the most The more popular the site, the more likely it blocks. Among top news websites that also rank in the 1,000 most visited sites on the web, 8 of 14 block GPTBot. In the top 5,000 it is 24 of 42, and in the top 10,000, 32 of 58, or 55.2%. Across all news publisher websites, the rate falls to 17.6%. For a brand, this means the outlets with the largest audiences are the ones most likely to be closed to OpenAI's training crawler. Coverage in those outlets still reaches readers, but whether ChatGPT can read and quote it depends on which of the three bots the outlet blocks. ## Blocking does not keep outlets out of ChatGPT Robots rules do not map neatly onto who gets cited. In a separate Suff Digital study of ChatGPT's news answers, Reuters was ChatGPT's most-cited outlet, in 250 answers, even though its robots.txt blocks ChatGPT-User and OAI-SearchBot. AP News was cited in 165 ChatGPT answers while blocking GPTBot. ChatGPT can cite a page it found through a search index or partner feed without fetching it, so a robots.txt block does not by itself remove an outlet from its answers. A robots.txt rule controls how a bot collects pages. On its own, it does not decide whether an outlet's name and stories appear in ChatGPT's answers, so a publisher can protect its archive from training and still be a leading source. ## What this means for publishers Decide about training and search separately. Blocking GPTBot keeps your content out of OpenAI's model training; blocking OAI-SearchBot and ChatGPT-User affects whether ChatGPT can fetch and cite your pages when it answers. Most top news sites that block one bot do not block all three, and each choice has a different cost in visibility. Review your robots.txt file at least once a quarter. New AI crawlers appear often, and many sites' rules name only a few of them, so the policy on paper can drift from the policy you want. Our technical seo services (https://www.suffdigital.com/seo/technical-seo-services) include a review of AI crawler rules for publishers and brands, matched to where you want to be cited. ## Embed this research Paste this on your site to embed the charts. It links back to the source automatically.

Source: 47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites - Suff Digital

## Cite this study APA Suff Digital. (2026). 47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites. https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot Plain link 47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites - Suff Digital - https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot ## Frequently asked questions ### How many news sites block ChatGPT's crawler? ### Do news sites block ChatGPT search too? ### Which major news sites block all of OpenAI's bots? ### Can a news site that blocks AI crawlers still be cited by ChatGPT? ## Related studies - AI SEO Reuters Is AI's Most-Cited News Source, in 328 AI News Answers: https://www.suffdigital.com/resources/data-studies/most-cited-news-outlets-ai - AI SEO ChatGPT Cites OpenAI's News Partners 2.2x as Often as Google News Does: https://www.suffdigital.com/resources/data-studies/chatgpt-cites-news-partners - AI SEO Paywalls in AI: Paywalled Outlets Are Cited in 16.7% of AI News Answers: https://www.suffdigital.com/resources/data-studies/paywalled-outlets-ai-news-answers - AI SEO Brands That Block GPTBot Still Get Named by ChatGPT: 31.5 Mentions vs 20.7 for Brands That Allow It: https://www.suffdigital.com/resources/data-studies/blocking-gptbot-ai-recommendations ## Structured data ```json [ { "@context": "https://schema.org", "@graph": [ { "@type": "Organization", "@id": "https://www.suffdigital.com/#organization", "name": "Suff Digital", "alternateName": "Suff Digital Marketing", "url": "https://www.suffdigital.com/", "logo": { "@type": "ImageObject", "@id": "https://www.suffdigital.com/#logo", "url": "https://www.suffdigital.com/logo.png", "contentUrl": "https://www.suffdigital.com/logo.png", "caption": "Suff Digital" }, "image": { "@id": "https://www.suffdigital.com/#logo" }, "description": "Suff Digital is a full-stack digital marketing agency that turns websites into predictable revenue engines, SEO, paid media, content, CRO, and web design built around revenue, not vanity metrics.", "slogan": "Digital marketing built for what's next.", "email": "hello@suffdigital.com", "telephone": "+1-629-236-0099", "foundingDate": "2022", "founder": { "@type": "Person", "name": "Matt Suffoletto", "jobTitle": "Founder & CEO" }, "contactPoint": [ { "@type": "ContactPoint", "telephone": "+1-629-236-0099", "email": "hello@suffdigital.com", "contactType": "sales", "areaServed": "US", "availableLanguage": [ "en" ] } ], "address": { "@type": "PostalAddress", "addressLocality": "Nashville", "addressRegion": "TN", "postalCode": "37219", "addressCountry": "US" }, "areaServed": { "@type": "Country", "name": "United States" }, "sameAs": [ "https://www.linkedin.com/company/suff-digital/", "https://www.instagram.com/suffdigital/", "https://www.youtube.com/@suffdigitalmarketing", "https://www.tiktok.com/@suffdigital" ], "knowsAbout": [ "Search Engine Optimization", "Pay-Per-Click Advertising", "Paid Social Advertising", "Conversion Rate Optimization", "Web Design", "Digital PR", "Link Building", "Website Migration", "Website Maintenance and Support", "Technical SEO", "Local SEO", "AI Search Optimization" ], "hasOfferCatalog": { "@type": "OfferCatalog", "name": "Suff Digital Services", "itemListElement": [ { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Search Engine Optimization (SEO)", "url": "https://www.suffdigital.com/" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Pay-Per-Click (PPC) Advertising", "url": "https://www.suffdigital.com/services/ppc" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Paid Social Advertising", "url": "https://www.suffdigital.com/services/paid-social" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Conversion Rate Optimization (CRO)", "url": "https://www.suffdigital.com/services/cro" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Web Design", "url": "https://www.suffdigital.com/services/web-design" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Digital PR & Link Building", "url": "https://www.suffdigital.com/services/digital-pr" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Website Migration", "url": "https://www.suffdigital.com/services/website-migration" } }, { "@type": "Offer", "itemOffered": { "@type": "Service", "name": "Website Maintenance & Support", "url": "https://www.suffdigital.com/services/website-maintenance" } } ] } }, { "@type": "WebSite", "@id": "https://www.suffdigital.com/#website", "url": "https://www.suffdigital.com/", "name": "Suff Digital", "inLanguage": "en-US", "publisher": { "@id": "https://www.suffdigital.com/#organization" } } ] }, { "@context": "https://schema.org", "@graph": [ { "@type": "Article", "@id": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot#article", "headline": "47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites", "description": "47% of the news websites Google News shows most often block GPTBot, the crawler OpenAI uses to collect training material for ChatGPT, against 17.6% of all news publisher websites. Far fewer block the bots ChatGPT uses to fetch and cite pages when it answers.", "image": [ "https://www.suffdigital.com/images/suff-digital-share-16x9.jpg" ], "url": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot", "datePublished": "2026-09-24", "dateModified": "2026-09-24", "author": { "@type": "Person", "name": "Matt Suffoletto", "jobTitle": "Founder & CEO", "url": "https://www.suffdigital.com/who-we-are" }, "publisher": { "@id": "https://www.suffdigital.com/#organization" }, "mainEntityOfPage": { "@id": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot", "@type": "WebPage" }, "isPartOf": { "@id": "https://www.suffdigital.com/#website" } }, { "@type": "BreadcrumbList", "@id": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://www.suffdigital.com/" }, { "@type": "ListItem", "position": 2, "name": "Resources", "item": "https://www.suffdigital.com/resources" }, { "@type": "ListItem", "position": 3, "name": "Data Studies", "item": "https://www.suffdigital.com/resources/data-studies" }, { "@type": "ListItem", "position": 4, "name": "47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites" } ] }, { "@type": "Dataset", "@id": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot#dataset", "name": "47% of Top News Sites Block ChatGPT's Training Crawler vs 17.6% of All News Sites", "description": "47% of the news websites Google News shows most often block GPTBot, the crawler OpenAI uses to collect training material for ChatGPT, against 17.6% of all news publisher websites. Far fewer block the bots ChatGPT uses to fetch and cite pages when it answers.", "url": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot", "creator": { "@id": "https://www.suffdigital.com/#organization" }, "datePublished": "2026-09-24", "dateModified": "2026-09-24", "license": "https://creativecommons.org/licenses/by/4.0/", "isAccessibleForFree": true, "measurementTechnique": "Google News results; robots.txt files from the public HTTP Archive crawl" }, { "@type": "FAQPage", "@id": "https://www.suffdigital.com/resources/data-studies/news-sites-block-gptbot#faq", "mainEntity": [ { "@type": "Question", "name": "How many news sites block ChatGPT's crawler?", "acceptedAnswer": { "@type": "Answer", "text": "47% of the news websites that appear most often in Google News block OpenAI's GPTBot training crawler (September 2026), against 17.6% of all 52,326 news publisher websites." } }, { "@type": "Question", "name": "Do news sites block ChatGPT search too?", "acceptedAnswer": { "@type": "Answer", "text": "Fewer do. 21.3% of top news websites block OAI-SearchBot, the bot behind ChatGPT's search results, and 42.1% block ChatGPT-User, which fetches pages for live answers." } }, { "@type": "Question", "name": "Which major news sites block all of OpenAI's bots?", "acceptedAnswer": { "@type": "Answer", "text": "The New York Times, USA Today, NBC News, NPR and CNN block all three OpenAI bots: GPTBot, ChatGPT-User and OAI-SearchBot." } }, { "@type": "Question", "name": "Can a news site that blocks AI crawlers still be cited by ChatGPT?", "acceptedAnswer": { "@type": "Answer", "text": "Yes. Reuters blocks ChatGPT-User and OAI-SearchBot yet is ChatGPT's most-cited news outlet, in 250 news answers, and AP News appears in 165 ChatGPT news answers while blocking GPTBot. ChatGPT can cite a page it found through a search index or partner feed without fetching it." } } ] } ] } ] ```