Transformers from Scratch

The full series in one post: why attention replaced recurrence; every building block of the Transformer — intuition, formula, PyTorch code, a numpy reference and an honest FLOPs count; then the assembled model, trained on a toy task, with the memory costs and the KV-cache explained.

July 24, 2026 · 54 min · Eugenia Elistratova

Variational Autoencoder (VAE)

A deep dive into variational autoencoders: the ELBO and its derivation, the reparameterization trick, CVAE — and the discrete-latent line of work (VQ-VAE, VQ-VAE-2, DALL-E) that grew out of it. With self-check questions and a hands-on notebook.

July 24, 2026 · 62 min · Eugenia Elistratova