Elektrine lite

← Feed

@NathanHarvey@kolektiva.social

2025-09-03 15:09 UTC

@keenancrane Although it is common to think of autoencoders “seeking to reproduce their input as closely as possible”, this isn’t actually their design. The scoring function is not always (maybe not even typically these days) a distance / metric over the input space. We know it is impossible to generally reproduce exactly (the information content in this process is lossy), so we don’t require exactness, but also the information content typically has other important qualities. So modern autoencoders can use scoring functions that capture these qualities. For example, in an image autoencoder, is the semantic content of the input found in the output? If the AE gets as input a picture of a cat walking on a wooden fence with dense trees and plants behind the wall, then you’d often like to ensure that the loss didn’t remove the semantic content (cat on wooden fence with dense trees and plants behind). But it would be acceptable to lose more pixel exactness as long as the semantic similarity is there. You don’t need to place each leaf or plank exactly, but don’t forget the cat. So it’s important when reasoning about AEs to make sure you aren’t building metric expectations. Scores don’t have to obey the triangle inequality, for example. I thought the reason that AEs are typically represented using trapezoidal encode/decode components is to show the relationship with other models like GANs which connect the components in the opposite order (to look like a hexagonish shape). You get a snapshot to reason about their training differences.

Replies (0)

No replies.