@zaire they don't listen to robots.txt anyway and those are usually directed at them so minus well make a file they listen to