Post #4298741

I have been thinking recently about how language is a high entropy latent space and what it means. Latent spaces are sort of magic: they don't just reduce the dimensionality of a higher dimensional space, they also constrain it such that arbitrary movement in the latent space maps to some *meaningful* position in the higher dimensional space. Think about how almost any textual prompt you give to an image generator results in a picture with recognisable shape and form, yet the vast majority of the possible images are basically just random noise.