@jwildeboer@social.wildeboer.net
Post #4433545
2026-08-07 11:56 UTC
Can someone from non-EU countries check https://www.crawlstop.com/robots.txt and post a screenshot of what they get? I notice that I (from Germany) get this EU specific part and I wonder if you get something geared towards your jurisdiction.
Replies (1)
-
@pitrh@mastodon.social 2026-08-07 12:02
@jwildeboer@social.wildeboer.net This is what I get from Norway (Telia Norway network): https://nxdomain.no/~peter/crawlstop_Norway_telia_network.jpg In the EEA, so likely same thing