Lobste.rs

An update on the scraper situation

barepenguin-70

LWN needs you LWN counts on its subscribing readers to support its mission of creating relevant human-written news coverage of the Linux and free-software communities. Please subscribe to LWN and help to keep us on the net. As a special offer, subscribe to LWN now for at least six months, and receive a 25% discount on your subscription. [+] Proceed to the article By Jonathan CorbetJuly 10, 2026 Our article “Fighting the AI scraper bot scourge”, published in early 2025, discussed the problem of widespread scraping of web sites in search of training data for large language models and related projects. This activity overwhelms sites with traffic. Over a year after that article is published, the problem is still growing. The hammering of sites by shadowy actors has reached new…

Continue reading at Lobste.rs lwn.net
Advertisement