Post #3474282

@doctormo@floss.social a technique I use which is simple and effective, but not sure if scales to your size. Just fail2ban and a honeypot of 404s. Create an page somewhere only a scraper will find. Have that page link to a bunch of missing (404) local URLs. Don't make it obvious. Configure fail2ban to read the access log for excessive 404s within a short period of time and the AI bot will block itself. Ensure fail2ban has scaling block times configured (bantime.increment = true). Trap (404|444|403|400).