Post #814903

Building a crawler is more complicated than I thought, One important thing that may be forgotten is outbound rules so the crawler don't try to crawl internal stuffs. #buildinpublic #webdev #indiedev