Google, OpenAI, Microsoft, Anthropic back one-line AI training opt-out

Key Points

1. Four AI providers agreed to honor a single robots.txt signal blocking generative training use.
2. The code, __usage_ai-training, replaces the need to list each company's crawler separately.
3. Site owners gain one-step refusal, but most providers only implement it during 2027.

The latest:

A single line in robots.txt will soon let any website refuse to have its content used for generative AI training across four major providers at once. Google, Microsoft, Anthropic and OpenAI announced support on 30 October 2026 for the policy token __usage_ai-training, ending the need to block each firm’s crawler individually. Anthropic has already begun implementing it.

Details:

• The syntax: Site owners write User-Agent: __usage_ai-training followed by Disallow: / in robots.txt, with optional carve-outs via Allow. The token is a usage policy label rather than a crawler name, so one entry covers every participating provider instead of a separate rule per company.
• Google’s timeline: Google said in its Search Central blog it intends to support the signal in early 2027. If a site refuses training through the token without naming Google-Extended explicitly, Google will treat that as a Google-Extended block, and its crawler documentation has been updated to flag the coming support.
• Google’s tools stay: Google-Extended and the generative AI opt-out in Search Console remain in force alongside the new signal. Google noted that sites permitting Google-Extended can appear in Gemini app answers and receive grounded clicks, framing the choice as a visibility trade-off rather than a pure privacy switch.
• OpenAI’s position: In a post titled Helping websites express AI training preferences, OpenAI said it will respect the signal and implement it within the coming months, publishing details then. GPTBot, OAI-SearchBot, OAI-AdsBot and ChatGPT-User remain operational alongside it.
• The limits: OpenAI stressed the signal is a coordination mechanism, not a security tool under RFC 9309, and does not stop non-compliant crawlers. Updating robots.txt also removes nothing already collected into existing datasets or models trained on it.
• Microsoft’s addition: Microsoft said on the Bing blog it will support the signal and add a separate AI Grounding setting, covering cases where a model reaches a page to answer a question rather than to gather training data. It extends to third-party tools built on Microsoft technology.
• Conflicting dates: The Verge reported Microsoft’s date as 1 February 2027, while SiliconANGLE described it as within the coming months. The Verge also reported that Meta expressed support for the idea without committing to implementation.
• The standards gap: The basis is an individual IETF draft, Machine-Readable Training Preference Signal, written by Kevin Kelley of Anthropic on 30 October 2026 and expiring 3 May 2027. IETF notes individual drafts carry no official standing in the standards process.
• The vocabulary: Its terms draw on the AIPREF working group draft by Paul Keller of Open Future and Martin Thomson of Mozilla. AIPREF is chaired by Mark Nottingham of Cloudflare and Suresh Krishnan of Cisco, with participants including News Corp, the Financial Times, BBC, EFF and Creative Commons.
• The caveats: SiliconANGLE reported the signal covers training only, not crawling, indexing or real-time retrieval, and excludes models performing exclusively non-generative tasks. Nate Hecht, an IETF participant, told Search Engine Roundtable the standard is unfinished and substantive disagreements persist.

Background:

Google said the proposal aligns with the European code of practice for general-purpose AI model providers, which recognizes robots.txt as a channel for expressing machine-readable preferences.

Between the lines:

The gap between announcement and enforcement is the story. Anthropic has started, Google is targeting early 2027, OpenAI gives no date, and Meta has not committed — so a site owner writing the token today is refusing four companies on paper and fewer in practice. Google’s reminder that Google-Extended permission brings Gemini visibility and grounded clicks prices the opt-out: refusal costs traffic.

What’s next

Watch Microsoft’s reported 1 February 2027 date, Google’s early-2027 rollout, OpenAI’s promised implementation details, and whether the individual IETF draft advances before it expires on 3 May 2027.