Post #1477109

They perform projections in a (very!) high-order smooth differentiable space. For problems which that describes, they work excellently. But when the problem doesn't fit that pattern, they're not so great. (The underlying math of hypersurfaces doesn't require differentiability, but the training algorithm guarantees it.) Public LLMs have an awfully large training dataset. Much bigger than you think.