From Autoencoder to Beta-VAE

An autoencoder learns an internal code by trying to reconstruct its input. A denoising autoencoder makes that exercise more demanding: it receives a damaged input but is trained against the original, clean version. Lilian Weng's 2018 tutorial describes adding noise or masking input values before reconstruction.

The encoder turns the damaged observation into a representation, and the decoder tries to recover the clean data from it. Simply copying the supplied input would reproduce the damage, so the training objective encourages recovery of structure despite the corruption. This provides a concrete way to train representations for robustness using targets derived from the original data.

“A nice byproduct is dimension reduction: the bottleneck layer captures a compressed latent encoding.”