Post #3654865
2026-07-07 07:15 UTC
RE: https://tooting.ch/@greg_harvey/116834354946383572
I said I would write up my findings on bot traffic to a major client of ours. Here it is!
https://gregharvey.gitlab.io/blog/posts/2026-07/fighting-the-bots/
Bots, #AI and in particular #ChatGPT - and what it means for modern website management. (Spoiler: It's not good.)
Replies (1)
-
@theeclecticdyslexic@mstdn.social 2026-07-07 18:13
@greg_harvey@tooting.ch The part I think people miss about scrapers is they don't only scrape for building training sets... but there is also scraping being done at inference time, because of RAG. You aren't just paying for them to steal your data once, you are paying for them to slightly improve their inference results as they offload the cost of maintaining the database they already assembled onto your server at runtime. This applies not only to frontier but also to local models!