Post #2676149
2026-04-23 20:25 UTC
@jsstaedtler@mastodon.art Just thinking out loud, but you could maybe catch the user agent of known crawlers in the PHP and send them a different page. Won't work for all crawlers, but would likely stop a chunk of them.
Replies (1)
-
@jsstaedtler@mastodon.art 2026-04-24 13:11
@jeff@higgy.social Happily, I'm able to use a .htaccess file in my root directory to configure Apache to redirect specific user agents. The frustration I have is that I'll discover a new one churning through my site, so I'll block them, but then a few days later *another* new one will come along to do the exact same thing 😮💨