Post #3287718

@zwol@masto.hackers.town @reykjalin@social.treehouse.systems @b0rk@social.jvns.ca Seems fairly similar to BERT, which is basically a predecessor to modern LLMs. Unsure of how much data you'd need to train something like that, transformer models in general are very resource hungry for training so they're kinda beyond what you can play with without datacenter GPU access.