Variational Autoencoder

A variational autoencoder learns to compress data into a compact space and rebuild it from there. The compressed space is probabilistic and smooth, which is what makes generating new samples possible.

An ordinary autoencoder learns fixed points, so the space between them is meaningless. A VAE encodes to a distribution instead, which forces the space to be continuous. Points near each other decode to similar things.

Think of it like this. Think of describing faces by a handful of dials rather than pixel by pixel. Because the dials vary smoothly, turning one slightly gives you a slightly different face rather than noise.

How it works

• The encoder outputs a distribution, a mean and a spread, rather than a single point.
• A sample is drawn from it, which is what forces nearby points to decode sensibly.
• The decoder reconstructs the original from that sample.
• Two losses balance. Reconstruction error pushes for accuracy, and KL divergence keeps the latent space well-shaped.

Trade-offs

• Stable training, blurrier output. The opposite balance to a GAN.
• A latent space you can navigate. You can interpolate between two inputs and get sensible things in between, which GANs do less reliably.
• Blur is inherent. Averaging over a distribution smooths detail, and that is a property of the design, not a bug to tune away.

Common mistakes

• "A VAE is an autoencoder." An autoencoder learns fixed points and cannot generate. The probabilistic encoding is the whole difference.
• "Reconstruction loss is enough." Without the KL term the latent space fragments and generation stops working.
• "VAEs make the best images." They do not. Diffusion models and GANs are sharper. VAEs give you stability and structure.