Post #2200403

@imrehg@fosstodon.org @danderson@hachyderm.io @ocean@raru.re Please don't run modified code on the production system, only run approved-by-AT code. Also, we tried to get what we could via https://wiki.archiveteam.org/index.php/Distributed_recursive_crawls (apparently not documented on the wiki page as of posting this)