OpenAI, Google, Microsoft, and Anthropic Unite Behind a Single AI Training Opt-Out Signal

By Hector Herrera | October 30, 2026 | News

The four largest AI developers have agreed on a shared robots.txt signal that lets website owners express a single AI training preference honored by all four platforms — ending years of fragmented, company-by-company opt-out requests that most publishers couldn't keep up with.

The unified signal, __usage_ai-training, is defined in an IETF Internet-Draft authored by Anthropic's Kevin Kelley and dated October 30, 2026. Anthropic has already begun implementing support. OpenAI and Microsoft are targeting early 2027. Google plans full support by end of 2027.

Why This Matters

Until today, a publisher who wanted to block AI training crawlers had to navigate a patchwork of per-company disallow rules — none of which were standardized, and most of which only covered specific bots, not model training at scale. A single misconfigured line meant your content could still be used. One token now covers all four companies at once.

How It Works

The new signal sits inside a website's robots.txt file — the same plain-text standard that search engines have used since 1994 to understand crawling permissions. Website operators add the __usage_ai-training directive to specify their preference:

• disallow — block all four platforms from using the content for training
• allow — explicitly permit training use
• No directive — current default behavior per each company's existing policies

Because the signal is defined in an IETF Internet-Draft rather than a proprietary spec, any AI company can adopt it — not just the four that signed on today.

The Timeline Gap Is Real

There is a meaningful gap between announcement and enforcement. Anthropic is implementing now. OpenAI and Microsoft follow in early 2027. Google arrives last, by end of 2027. Content scraped between now and those dates remains subject to each company's existing policies.

That gap matters to publishers. If a site adds the opt-out signal today and Google's crawler visits tomorrow, the signal won't be honored until Google's implementation is live — likely a year away.

What It Means for Publishers and Developers

For website operators: this is the closest thing to a universal opt-out that has ever existed. One robots.txt entry, four companies covered. The signal won't retroactively cover content already in training datasets, but it limits future exposure.

For developers building on these models: this signals the industry is moving toward structured consent frameworks rather than waiting for courts to define the rules. The IETF process gives the standard legitimacy beyond a handshake agreement.

For AI companies: joint adoption reduces the legal and reputational risk of training on contested data — and pre-empts regulatory mandates that are already moving through the EU and several U.S. states.

Context: The Scraping Wars

The backdrop is two years of escalating litigation. Publishers, book authors, and record labels have filed dozens of lawsuits arguing that AI training on copyrighted material without consent or compensation is infringement. Courts have been inconsistent: some have allowed training as fair use, others have found it actionable. The Suno lawsuit settlement in October 2026 showed that even major AI companies are willing to negotiate rather than fight every case to verdict.

A voluntary industry standard doesn't resolve those pending cases — but it changes the landscape for future content.

What to Watch

The IETF Internet-Draft is still in draft status. Watch for the standardization process to move toward an RFC (Request for Comments), which would give the signal formal status and encourage broader adoption beyond the founding four. Also watch whether major CMS platforms — WordPress, Squarespace, Wix — build the opt-out into their default templates, which would dramatically increase adoption by non-technical publishers.

Source: OpenAI announcement