Are ISPs responsible for bots having residential IPs or is this a user problem?

2026-06-22 10:53 UTC

I’m trying to understand the bot problem in the internet and finding more ways to defend myself. One thing that I can’t seem to understand is why most bots, scrapers and crawlers seem to have residential IPs. Is it that ISPs are being paid by tech-bros to assign them these IPs? Is it that residential devices have been hacked /contain malware that does this? Is it trivial for companies to assign themselves residential IPs? Paid volunteers are doing this for AI companies? Or is there is some other reason for this? Obviously this is a problem because one can rotate / cycle through residential IPs and if I aggressively block each offender in my logs permanently, then the next person assigned this IP who may be a legitimate user will be unable to access my site.

Replies (3)

@Mordikan@kbin.earth 2026-06-22 15:42

I worked as a network analyst for a provider for several years and during that time I'd say ~90% of the issue stemmed from sketchy apps/services that the user loaded from their end. A lot of "free" VPN services will basically allow bad actors (the paid tier) to use your connection. A lot of IoT devices are also just openly available on the Internet to route through. From the ISP perspective, we managed the roads, not your car. There is a push to blame the ISP as it's their network, but realistically how are they meant to provide security (in the context that is being asked) to any device that gets plugged into that network? We even had business customers demand we add clauses to contracts where we would accept responsibility for any malware they sent between sites over an MPLS setup. In the end, a lot of people seem to want this impossible scenario of the ISP managing security for them but also not inspecting their traffic.

Open ##3491619
@OnfireNFS@lemmy.world 2026-06-23 07:08

You should consider looking into fail2ban or Crowdsec. I recently added Crowdsec to my firewall and it seems to work pretty well. I don’t host any public facing services on my network. But maybe someone else with more experience can chime in about it. I believe both tools are pretty good for stopping bots

Open ##3501110
@0x0@lemmy.zip 2026-06-22 14:30

Residential routers have crappy security 'cos no-one bothers to punish ISPs for crap security practices. Also some free VPNs come at a malware cost.

Open ##3502396