Artifipedia
GlossaryMapBlog
Home/Deep Learning/Autoencoder
Deep Learning

Autoencoder

A network trained to copy its input through a bottleneck — which forces it to learn what matters, and is the ancestor of most representation learning.

Reading level: Curious
Pick your depth ↓

When not to use it

  • For dimensionality reduction, by default. PCA is faster, deterministic, interpretable, and often equivalent. Use an autoencoder when the structure is genuinely non-linear.
  • For generation. A plain autoencoder's latent space has holes. Sampling from it decodes to garbage. That's what VAEs are for.
  • On dirty data, for anomaly detection. If anomalies are in the training set, the model learns to reconstruct them and they become invisible.
  • When you don't have much data. You're learning a compression scheme; that needs examples.

Reach for something else instead

  • PCA — for linear structure, which is more often enough than people expect.
  • VAE — when you need the latent space to be generative.
  • Pretrained embeddings — usually better than an autoencoder you trained yourself.
  • UMAP — for visualisation specifically.

Sources & further reading

  • Baldi & Hornik (1989), Neural Networks and Principal Component Analysis — the linear autoencoder learns PCA; the result that bounds what autoencoders add.
  • Hinton & Salakhutdinov (2006), Reducing the Dimensionality of Data with Neural Networks — deep autoencoders beating PCA; part of what revived the field.
  • Vincent et al. (2008), Extracting and Composing Robust Features with Denoising Autoencoders — denoising as the better objective, and the ancestor of diffusion.

Primary sources, listed so you can check the claims on this page rather than take them on trust.

Where people go wrong

  • Making the bottleneck too wide, so the network learns to copy and compresses nothing.
  • Expecting to sample from the latent space. Plain autoencoders aren't generative.
  • Using one where PCA would do. A linear autoencoder is PCA — you've added complexity for nothing.
  • Training an anomaly detector on data containing anomalies, which teaches the model they're normal.

At a glance

FieldDeep Learning
Ideareconstruct through a bottleneck; the compression is the point
Linear versionis PCA (Baldi & Hornik, 1989)
Not generativelatent space has holes
Real usesanomaly detection, denoising, the compression stage in diffusion
Live frontiersparse autoencoders for interpretability
DifficultyIntermediate
Flashcards for this concept
Question
—
Answer
—
1 / 4

Often compared with

Autoencoder vs. VAE — one learns a compression, the other learns a compression whose space you can sample from. Only the second can generate.

Related concepts

Deep Learning

Neural Network

A system of simple connected units that learns patterns from examples — the foundation underneath deep learning and modern AI.

Generative AI

Variational Autoencoder

An autoencoder whose latent space you can actually sample from — the principled generative model that lost to GANs on looks and won by being useful.

Machine Learning

Dimensionality Reduction

Squashing many features into few — useful for compression and computation, and dangerous the moment you believe the picture.

Generative AI

Latent Space

The compressed space a model thinks in — where similar things sit close together, and where the famous vector arithmetic works better in demos than in practice.

Applied AI

Anomaly Detection

Finding the unusual thing — where the base rate makes precision nearly impossible and almost every deployment drowns in false alarms.

Deep Learning

Transformer

The neural-network architecture behind almost every modern AI model — built around attention, which lets it weigh every word against every other, all at once.