Post #2686063
2025-05-12 12:37 UTC
@tdp_org@mastodon.social take a look at https://datatracker.ietf.org/doc/html/draft-meunier-web-bot-auth-architecture-01
Replies (1)
-
@tdp_org@mastodon.social 2025-05-12 13:01
@SimmerVigor@mastodon.social Oh nice! From the title and ToC that looks very much like the sort of thing I've been ½ mulling...🙌🏻 I also feel like we need to move to more of an intent/purpose-based model of identification/robots.txt blocking. The current system is way too onerous to maintain. Would be a lot better to be able to say e.g.: * Search Engine bots: Yes * LLM data collectors: No etc. Obv that relies on well-mannered bot operators but we do right now too.