Post #1757866

@aredridel > It's that agent _users_ and the people serving them have a very large demand to access information with semi-automated systems Which information though? What kind of agent user could have demand to access information on git blame page of some obscure file in one of my repos on my obscure gitea insurance? > It's not that the big model companies are scraping the web and ignoring robots.txt I heard that big model companies outsource the scraping to the lowest bidders.