search
HomeTechnology peripheralsIt IndustryReddit was revealed to have blocked multiple search engines and AI crawlers, and officials said it had nothing to do with Google's cooperation

News from this site on July 25. In February this year, Google announced a new agreement with Reddit. Reddit data will be used to train Google’s AI model and highlight Reddit results in Google search results. Bloomberg reports the deal is worth about $60 million. Since then, Reddit has also seen a significant increase in impressions on Google searches. The latest news is that Reddit has banned other search engines from retrieving its website content. Foreign media 404 Media first reported this behavior and pointed out that Reddit has updated its robots.txt file (this site’s note: the agreement file between the website and the crawler) to prevent all robots from crawling any content on the site. This site inquired about the robots.txt file of the Reddit website and learned that the platform stated that "Reddit believes in an open Internet, but does not believe in the abuse of public content." Its "Disallow" option is set to "/", which means that retrieval of content in the root directory of the website is prohibited. All files.

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

The document was first updated in June of this year, with Reddit explaining that the change was due to "a significant increase in commercial entities scraping Reddit content and misuse of the content."

Foreign media 9To5Google said that search engines such as Bing, DuckDuckGo, Mojeek and Qwant have all been affected. When searching for content using "site:reddit.com", either the latest content or complete website results are not displayed. .

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

However, in the latest test of this site, Bing and DuckDuckGo have re-displayed the search content of "site:reddit.com".

Reddit 被曝屏蔽多个搜索引擎和 AI 爬虫,官方称与谷歌合作无关

Reddit spokesperson Tim Rathschmidt said in a statement to The Verge:

This is completely unrelated to our recent partnership with Google .

We have been in discussions with multiple search engines, but have been unable to Come to terms with everyone because some people are unable or unwilling to make enforceable commitments regarding their use of Reddit content, including their use of AI training.”

The above is the detailed content of Reddit was revealed to have blocked multiple search engines and AI crawlers, and officials said it had nothing to do with Google's cooperation. For more information, please follow other related articles on the PHP Chinese website!

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Top 21 Developer Newsletters to Subscribe To in 2025Top 21 Developer Newsletters to Subscribe To in 2025Apr 24, 2025 am 08:28 AM

Stay informed about the latest tech trends with these top developer newsletters! This curated list offers something for everyone, from AI enthusiasts to seasoned backend and frontend developers. Choose your favorites and save time searching for rel

Serverless Image Processing Pipeline with AWS ECS and LambdaServerless Image Processing Pipeline with AWS ECS and LambdaApr 18, 2025 am 08:28 AM

This tutorial guides you through building a serverless image processing pipeline using AWS services. We'll create a Next.js frontend deployed on an ECS Fargate cluster, interacting with an API Gateway, Lambda functions, S3 buckets, and DynamoDB. Th

CNCF Arm64 Pilot: Impact and InsightsCNCF Arm64 Pilot: Impact and InsightsApr 15, 2025 am 08:27 AM

This pilot program, a collaboration between the CNCF (Cloud Native Computing Foundation), Ampere Computing, Equinix Metal, and Actuated, streamlines arm64 CI/CD for CNCF GitHub projects. The initiative addresses security concerns and performance lim

Building a Network Vulnerability Scanner with GoBuilding a Network Vulnerability Scanner with GoApr 01, 2025 am 08:27 AM

This Go-based network vulnerability scanner efficiently identifies potential security weaknesses. It leverages Go's concurrency features for speed and includes service detection and vulnerability matching. Let's explore its capabilities and ethical

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools