Post #2337191
2026-03-13 14:13 UTC
@ziadkh0@fosstodon.org It does that when LLM crawlers come through and I just don’t want to spend money to increase server bandwidth so they can continue to scrape it. It’s a bummer.
Replies (1)
-
@ziadkh0@fosstodon.org 2026-03-13 14:22
@aardrian@toot.cafe That is indeed a bummer 😞 I remember Cloudflare added an option to block AI crawlers*. I haven’t used it yet, but might be worth a shot 🤔 *: https://www.cloudflare.com/learning/ai/how-to-block-ai-crawlers/