Post #2676140
2026-04-23 16:11 UTC
@jsstaedtler@mastodon.art a dumb solution would be to tell robots to not index the page (robots meta tag) if there is any tag queries, which i assume you can do via PHP.
edit: or if you want individual tags indexed, at least reject robots for queries of more than one tag?
Replies (2)
-
@rubenwardy@hachyderm.io 2026-04-23 16:18
@redstrate@mastoart.social @jsstaedtler@mastodon.art Many crawlers ignore this in my experience, especially the AI ones
-
@jsstaedtler@mastodon.art 2026-04-23 16:25
@redstrate@mastoart.social Ah, this sounds promising! I don't want to make my site invisible on the greater Web by blocking all bot crawlers, but I'd be fine with them only loading URLs with no queries/parameters (anything after a ?). I'll look into that meta tag, though I acknowledge the other reply here that bots can happily ignore that.