@Rhube@wandering.shop @eliasulrich@hachyderm.io Developing your own crawler can be time consuming and expensive. Bing which (ddg and many others use) is far worse. Using Google to pull data and then using logic to sort through it isn’t in itself a problem. Heck most SearXng instances I run into still use a Google crawler. Using an LLM to parse through tons of data can be effective. It’s all about how it’s trained. Websites can be all over the place. Trying to manually code to account for it all will leave huge gaps imo Overall I’ve been impressed with much of its search results. Plus the code is open source.