#robotstxt
-
OpenAI, Google, Microsoft and Anthropic back a shared signal for opting out of AI scraping
The biggest AI companies have agreed to honor a common "policy token" in robots.txt that lets publishers block the use of their content for model training in one step.
-
Meta's new terms may let it train AI on news sites for free
Meta's revised commercial terms, taking effect on October 30th, appear to let the company crawl websites "associated" with business accounts, including news publishers, and use their content to train its AI.