Cyberstern
dunkelstern@corteximplant.com
<p>Crazy inventor. on arch btw, sometimes a crab.</p><p>Will build stuff for coffee.</p><p>Find me pounding servers in the basement.</p><p>If my instance at kampftoast.de should be down you can find me here.</p>
Posts
-
Post #995370
Thinking of building a semantic search engine based on vector embedding (SBERT, probably self trained models in the future based on what was indexed to disconnect from the bigtech stuff) and parallel full text search. What language to use? It will be separated into 3 parts: Crawler/Data importer (Importing public data dumps, well behaved RSS and HTTP crawler)Parser (Splitting paragraphs/sentences, calculating embedding vectors, keyword extraction),Frontend (Web interface/API, full-text-search...