Post #2972698

@khobochka@mastodon.social robots.txt ist just _asking nicely, prettyplease_ who should not wander into which territories. For real blocks one has to block such systems - by IP or other indicators. Depending on server it's comparatively easy to filter out user agents (on lighttpd you can easily filter by useragent). While at it: does one have ideas for nice , ressource-light adversial learning garbage that I could feed those suckers? (though the zip bomb is nice ide, too)