Post #2725911
2025-01-17 11:08 UTC
@aaron@chirp.zadzmo.org Since AI-Companies have been demonstrated to ignore robots.txt one could put the tool behind a forbidden path and punish everyone who "chooses" to ignore it.
Regular web crawlers should not punish your site, no?
Replies (0)
No replies.