Post #4202813
2026-07-29 14:02 UTC
@lcamtuf@infosec.exchange I just have a metric telling me that they crawled 5000 pages on my site, but decided 90% were too garbage to index for some reason? not sure which is more painful
Replies (1)
-
@lcamtuf@infosec.exchange 2026-07-29 15:16
@jmason@mastodon.ie I think that's sort of normal, it's usually some combination of broken (e.g., mistyped) external links, duplicate ways to reach the same content, redirects, exclusions, etc. For example, this is for my homepage.