Post #2676136
2026-04-23 15:59 UTC
@jsstaedtler@mastodon.art an easy way to catch this is that these scrapers generally don't send Referer headers, so you can kill these by checking that a valid Referer header is present in tag search. This will have false positives for humans that try to be too smart though.
Replies (2)
-
@oblomov@sociale.network 2026-04-23 16:05
@jsstaedtler@mastodon.art (talking from experience with my self-hosted gitweb for this, BTW)
-
@lumi@snug.moe 2026-04-23 16:07
@oblomov@sociale.network @jsstaedtler@mastodon.art the referer header only exists for tracking, so many privacy-conscious people configure their browsers not to send it the referer header should not exist in the first place