Diffusion probabilistic models (Sohl-Dickstein 2015): Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Add noise to data in many small steps until nothing is left, then train a network to undo the steps one at a time.

Each small noising step has a Gaussian reverse, so the network only has to predict a mean and a variance per step. Run the learned reverse steps from pure noise and you get samples; average the log of the ratio of reverse to forward step probabilities along a noising path and you get a lower bound on the log-likelihood.

A drop of ink spreads through a glass of water until the glass is evenly gray. The spreading is easy to simulate and very hard to run backward. This 2015 paper shows that if the spreading happens in small enough steps, each backward step is simple enough for a neural network to learn.

This is the paper that introduced diffusion models. DDPM, score-based SDEs, Stable Diffusion and the diffusion policies used in robotics all keep its three parts: a fixed process that turns data into noise, a learned process that turns noise back into data, and an objective that ties the two together. Several details that are now standard (predicting the noise, the squared-error loss, the $\bar\alpha_t$ formula for jumping to any noise level in one step) came five years later, and this page marks which is which.

The argument has five pieces, taken in order below: why flexible probability models are usually intractable; the forward process that dissolves data into a Gaussian; why the reverse of a small noising step is itself Gaussian, so a network can learn it by regression; the bound $K$ that trains the network and also reports a likelihood; and how the same chain can be conditioned on an observation to inpaint or denoise an image.

Flexible or tractable

A generative model is a probability distribution $p(\mathbf{x})$ fitted to data, say 32×32 color images. You want to do three things with it: draw samples, evaluate $p(\mathbf{x})$ for a given image, and fit it to a dataset. The paper calls a family tractable when those three operations are cheap and exact, and flexible when it can match the structure of real data.

A Gaussian is tractable: its density is a formula, sampling takes one call to a random number generator, and fitting it means computing a mean and a covariance. It cannot represent a dataset of handwritten digits, because digits are not an ellipsoidal cloud. The usual route to flexibility is an energy model, any non-negative function $\phi(\mathbf{x})$ divided by its total mass:

For an image, $Z$ is an integral over a 3072-dimensional space, and nobody can compute it for an expressive $\phi$. Without $Z$ you cannot evaluate $p$, and fitting or sampling falls back on Markov chain Monte Carlo, which is slow and only approximate.

The paper gets both properties by defining the model as the end of a long Markov chain. Each link in the chain is a Gaussian, so each link can be sampled and evaluated in closed form. The flexibility comes from the number of links: a chain of 1000 slightly different Gaussian steps can carry a standard Gaussian onto a very non-Gaussian distribution, and the abstract describes the result as a deep generative model "with thousands of layers or time steps".

The forward process: how diffusion destroys data

The chain is built from two processes. The forward process is fixed and needs no training: it takes a data point $\mathbf{x}^{(0)}$ and applies $T$ noising steps. Each step scales the current point down slightly and adds a little Gaussian noise:

The number $\beta_t$ is the variance of the noise added at step $t$, and it is small. The mean is the old point multiplied by $\sqrt{1-\beta_t}$, which is slightly less than 1. Shrinking the signal while adding noise keeps the variance fixed: if $\mathbf{x}^{(t-1)}$ has variance 1 per coordinate, then $\mathbf{x}^{(t)}$ has variance $(1-\beta_t)\cdot 1 + \beta_t = 1$. The paper scales every dataset except the binary one to variance 1, and the chain ends at the standard Gaussian $\pi = \mathcal{N}(\mathbf{0}, \mathbf{I})$ instead of growing without bound. Score-SDE (2021) later named this kind of chain "variance preserving"; the 2015 paper gives the kernel without a name.

With a constant $\beta = 0.01$, one step multiplies the signal by $\sqrt{0.99} = 0.995$. After 100 steps the original signal has been multiplied by $0.99^{50} = 0.61$, after 500 steps by $0.081$, and after 1000 steps by $0.0066$, at which point the sample is Gaussian noise with a trace of the image. The paper uses $T = 1000$ steps for its image models, except the bark textures ($T = 500$), and 40 steps for its two-dimensional swiss roll.

The schedule $\beta_1, \dots, \beta_T$ does not have to be constant. For Gaussian diffusion the paper learns $\beta_2, \dots, \beta_T$ by gradient ascent on the training objective, with $\beta_1$ fixed to a small constant to prevent overfitting; a footnote adds that a fixed schedule seems to work as well. For binary data it uses the fixed schedule $\beta_t = (T-t+1)^{-1}$, which removes the same 1/T fraction of the original signal at every step, so after $t$ of $T = 1000$ steps a fraction $(T-t)/T$ of it is left: half at step 500.

The figure runs the forward process on the paper's toy dataset. Drag the step from 0 to 1000 and watch the readout: the signal coefficient falls and the noise coefficient rises while the total variance stays at 1.00.

The figure jumps straight to step $t$. Composing $t$ Gaussian steps gives one Gaussian whose signal coefficient is the product $\sqrt{\prod_{s \le t}(1-\beta_s)}$, written $\sqrt{\bar\alpha_t}$ since DDPM. The paper's text never writes this marginal or the notation; it gives only the single step (1). The reference implementation released with it does use the cumulative product to sample $\mathbf{x}^{(t)}$ in one jump during training, as the code later on this page shows.

Why small steps make the reverse step Gaussian

To generate, the model has to run the chain backward: start from a draw of $\pi$ and undo one step at a time. Undoing step $t$ means sampling from $q(\mathbf{x}^{(t-1)} \mid \mathbf{x}^{(t)})$, the distribution over where a point was one step earlier given where it is now. That distribution depends on the data, so it is unknown and has to be learned.

The paper rests on a classical result it attributes to Feller (1949): for a diffusion with small enough steps, the reverse step has the same functional form as the forward step. The forward step is Gaussian, so for small $\beta_t$ the reverse step is also Gaussian, and a Gaussian is described completely by a mean and a covariance. Learning the reverse process therefore reduces to learning, for every step, a function that outputs a mean and a function that outputs a covariance. In the paper's words, "the task of estimating a probability distribution has been reduced to the task of performing regression on the functions which set the mean and covariance of a sequence of Gaussians."

The step has to be small because of Bayes' rule. The reverse step is proportional to the forward likelihood times the distribution of the earlier point:

The first factor is a Gaussian bump of width about $\sqrt{\beta_t}$ around $\mathbf{x}^{(t)}$. When $\beta_t$ is small, the bump is narrow, and over such a short range the log of the second factor is close to a linear function. A Gaussian times the exponential of a linear function is another Gaussian, shifted, so the product is still a single Gaussian bump. When $\beta_t$ is large, the bump is wide enough to cover several clusters of the data, and the product inherits their peaks.

The figure shows this for one-dimensional data with two clusters. The teal curve is the exact reverse distribution over the clean value given one noisy observation; the dashed curve is the closest single Gaussian; the readout is the KL divergence between them. At $\beta = 0.02$ the reverse is one narrow bump and the KL is 0.01 nats. At the default $\beta = 0.55$ it has split into two peaks, the best Gaussian sits in the empty valley between them, and the KL is 0.48.

This sets the trade-off behind the method. Small steps make every reverse step learnable by regression but require a long chain; large steps shorten the chain but make each reverse step multimodal, and a Gaussian reverse kernel then fits it badly. Diffusion takes the long chain. The model's reverse step is

where $\mathbf{f}_\mu$ and $\mathbf{f}_\Sigma$ are outputs of one network that reads the noisy point and the step index. The network outputs the reverse Gaussian's mean and its covariance (diagonal, for images) directly. It does not predict the added noise $\boldsymbol{\epsilon}$; that parameterization, with its simpler loss, is DDPM's.

Sampling: run the learned chain backward

Once $\mathbf{f}_\mu$ and $\mathbf{f}_\Sigma$ are trained, sampling is a loop. Draw $\mathbf{x}^{(T)}$ from $\mathcal{N}(\mathbf{0},\mathbf{I})$. At each step, evaluate the network to get a mean and a covariance, and draw $\mathbf{x}^{(t-1)}$ from that Gaussian. After $T$ steps, $\mathbf{x}^{(0)}$ is a sample from the model. A fresh Gaussian draw is made at every step, which makes this an ancestral sampler: each step samples from the conditional distribution given the step before, the way you sample a Bayesian network from its roots down.

The figure runs this loop on the swiss roll with 40 steps, the length the paper used for this dataset. The reverse steps here are the exact ones for this toy distribution, standing in for a trained network. The teal arrows are the drift $\mathbf{f}_\mu(\mathbf{x}, t) - \mathbf{x}$, the change in mean each step applies to a point, which the paper plots in the bottom row of its Figure 1. Press Play and watch the arrows and the readout. At $t = 40$ every arrow points at the center of the data and the longest one is 4.1 units, almost the full distance from the corner of the grid to the center. By $t = 10$ the arrows over the roll have shrunk, the outer ones still point inward, and the particles have spread around the spiral; by $t = 5$ most of them sit on its arms.

The drift changes because of how much signal is left. At $t = 40$ the noisy point says almost nothing about which part of the roll it came from, so the best estimate of the earlier point is close to the mean of the whole dataset and the step mostly replaces the input with fresh noise. As the noise level falls, a point carries enough information to place it near one arm, and the mean of the step moves it toward that arm instead of toward the center. The coarse layout of a sample is settled at high noise and the fine detail at low noise.

Switch the fresh noise off and every particle follows the mean of each step. The first reverse step from $t = 40$ forgets almost everything about its input, so all 110 particles start the second step at nearly the same point, follow the same means, and end together on one spot. The spread of the samples comes from the noise drawn with variance $\mathbf{f}_\Sigma$ at every step.

How diffusion models are trained: the bound K

Training needs an objective. The natural one is the log-likelihood of the data, but the model's probability of a clean point is an integral over every path the reverse chain could have taken to reach it:

For images that integral runs over 1000 × 3072 dimensions. The paper borrows a trick from annealed importance sampling: multiply and divide inside the integral by the forward path probability $q(\mathbf{x}^{(1\cdots T)} \mid \mathbf{x}^{(0)})$, which is known exactly. The likelihood becomes an average, over forward paths started at the data point, of the ratio between reverse and forward step probabilities. Forward paths are cheap to sample. The log of an average is at least the average of the log (Jensen's inequality), which gives a lower bound:

$K$ is a lower bound on the log-likelihood, so training maximizes $K$; the loss handed to the optimizer is $-K$. The paper notes that the derivation parallels the bound used in variational Bayesian methods, the same shape of argument that trains a variational autoencoder.

Appendix B of the paper applies Bayes' rule to the forward process and rewrites (3) as a sum of KL divergences plus entropy terms, all of which can be computed in closed form:

The sum starts at $t = 2$ and carries a minus sign, so maximizing $K$ means shrinking every KL term. The three entropies at the end depend only on the forward schedule and the prior, not on the network. Inside each KL, the network's reverse step $p(\mathbf{x}^{(t-1)} \mid \mathbf{x}^{(t)})$ is compared with $q(\mathbf{x}^{(t-1)} \mid \mathbf{x}^{(t)}, \mathbf{x}^{(0)})$, the reverse of the forward process when the clean starting point is also known. That conditioning makes the target computable: given $\mathbf{x}^{(0)}$, the pair $\mathbf{x}^{(t-1)}$, $\mathbf{x}^{(t)}$ is jointly Gaussian, so the target is a Gaussian with a closed-form mean and variance. The KL between two Gaussians is also closed form, so every term in (4) is analytic, and one training step can score a single randomly chosen $t$.

The figure computes $K$ exactly for a one-dimensional dataset of two clusters, scaled to variance 1, where the true reverse steps can be written down. Each bar is one step's KL between the true reverse step and the model's Gaussian, split in two. The violet part is the error of an imperfect kernel, which training removes. The amber part is what remains when the kernel is the best possible Gaussian, the cost of approximating a non-Gaussian reverse step by a Gaussian. The bottom bar measures $K$ against the baseline the paper reports in its Table 1, $L_{\text{null}}$, the log-likelihood of the plain Gaussian $\pi$. For this dataset no model can beat that baseline by more than 0.740 bits.

Read three settings off the figure. With $T = 1$, $K$ falls short of its ceiling, $L_{\text{null}}$ plus 0.740 bits, by the full 0.740 bits at every training level: a single Gaussian step from noise to data can at best produce a Gaussian, and the best Gaussian for variance-1 data is $\pi$ itself. With full training the shortfall is 0.060 bits at $T = 10$, 0.006 at $T = 40$, and under 0.001 by $T = 1000$. With training at 0 the shortfall is 0.740 bits for every $T$, because that untrained chain reproduces $\pi$ exactly.

One step of training on a real example looks like this. Take a CIFAR-10 image as a vector of $32 \times 32 \times 3 = 3072$ numbers, add uniform noise to undo the 8-bit pixel quantization, and scale to variance 1. Pick $t = 500$ of $T = 1000$. Under the reference code's schedule about half the original variance is left at step 500, so

The network reads $\mathbf{x}^{(500)}$ and the step index and returns a 3072-dimensional mean and a 3072-dimensional diagonal variance for $\mathbf{x}^{(499)}$. The closed-form target gives another mean and variance. The per-pixel Gaussian KL between the two, averaged over pixels and images and multiplied by $T$ to stand for the sum over steps, is the estimate of $-K$ that gets backpropagated:

# One training step, CIFAR-10, after model.py in the reference code.
# x0: (n, 3072) images with uniform dequantization noise, scaled to variance 1.
t     = randint(1, T)                  # ONE step per minibatch, T = 1000
abar  = prod(1 - beta[:t + 1])         # signal variance left at step t
eps   = randn_like(x0)                 # frozen noise: held fixed for the gradient
xt    = sqrt(abar) * x0 + sqrt(1 - abar) * eps        # jump straight to step t
mu_q, var_q = forward_posterior(x0, xt, t)  # q(x_{t-1} | x_t, x0), closed form
z_mu, z_var = net(xt, t)               # multi-scale conv net, (n, 3072) each
mu_p  = xt * sqrt(1 - beta[t]) + z_mu * sqrt(beta[t])
var_p = sigmoid(z_var + logit(beta[t]))     # reverse variance near beta[t]
kl    = gauss_kl(mu_q, var_q, mu_p, var_p)  # (n, 3072), analytic per pixel
neg_K = T * kl.mean() + entropy_terms       # one-sample estimate of -K
neg_K.backward(); opt.step()                # descend -K = ascend the bound K

The noise $\boldsymbol{\epsilon}$ is drawn once and held fixed while the gradient is taken, which the paper calls frozen noise and credits to Kingma and Welling. Holding $\boldsymbol{\epsilon}$ fixed makes $\mathbf{x}^{(t)}$ a differentiable function of the schedule, and that is how the paper's Gaussian runs get a gradient for $\beta_2,\dots,\beta_T$. In the reference code the schedule is fixed, so the gradient reaches only the network. Sampling needs no data and no target; it runs the learned chain from noise:

# Sampling: ancestral rollout of the learned reverse chain (sampler.py).
x = randn(n, 3072)                     # start from the prior pi = N(0, I)
for t in range(T - 1, 0, -1):          # walk the noise level down
    mu, sigma = net_mu_sigma(x, t)     # learned reverse mean and std
    x = mu + sigma * randn(n, 3072)    # fresh noise at every step
return x                               # one sample per row

The network and the reference code

The paper used three networks. The swiss roll model is a radial basis function network with one hidden layer of 16 units, shared across all 40 steps, with a separate readout layer per step; its variance output passes through a sigmoid to stay between 0 and 1. The binary sequence model is a multilayer perceptron with three hidden layers of 50 units that outputs the Bernoulli rates for 2000 steps of binomial diffusion. The image models use a multi-scale convolutional network: each layer mean-pools the image by powers of two, convolves at every scale, upsamples, and sums, which gives a full-resolution output that still sees long-range structure.

The image network does not output the mean and variance directly. For each pixel it outputs two vectors of coefficients that weight a set of bump functions of $t$, producing per-pixel values $z^\mu$ and $z^\Sigma$ that vary smoothly across steps. Appendix D then turns them into the reverse step as a perturbation of the forward kernel:

The function $\sigma$ is the logistic sigmoid. With $z^\Sigma = 0$ the reverse variance equals the forward $\beta_t$, so the network starts from a sensible guess and learns a correction. The mean moves the pixel a fraction $\Sigma_{ii}$ of the way toward $z^\mu_i$: a pixel at $x_i = 0.8$ with $z^\mu_i = 0.2$ and $\Sigma_{ii} = 0.01$ gets mean $0.6 \times 0.99 + 0.2 = 0.794$. The paper describes $z^\mu$ as the mean of the distribution you would reach by applying the same reverse step many times.

The public reference implementation, which the paper says produced the CIFAR-10 results, differs from this in a few places. Its active mean is $x\sqrt{1-\beta_t} + z^\mu\sqrt{\beta_t}$, with a form close to (5) left commented out. It fixes $\beta_t$ at a schedule that removes a constant fraction of the original variance per step, with learning it marked as a to-do. It samples one step $t$ per minibatch and multiplies that step's KL by $T$, as in the code above (file and line numbers are in the Provenance panel at the end).

Where the thermodynamics comes in

The title refers to nonequilibrium statistical physics. The forward process is a diffusion in the physical sense: it moves a system from a structured state to the featureless equilibrium $\pi$. The paper borrows two results about such processes.

The first is the likelihood estimator. The average of a reverse-to-forward path ratio in (3) is the weight used by annealed importance sampling (Neal, 2001), a method for estimating an intractable normalizing constant by moving gradually from a simple distribution to a hard one. The Jarzynski equality (1997) is the physics version of the same identity: averaging the exponentiated work over many fast, irreversible runs of a process gives the equilibrium free-energy difference exactly. The paper cites both as the source of its approach. This page adds an analogy the paper does not make: Jensen's inequality in (3) plays the part of the second law of thermodynamics, which says the average work done on a system is at least its free-energy change.

The second is when the bound becomes exact. A process run slowly enough to stay in equilibrium at every moment is called quasi-static, and it is reversible: the forward and reverse path distributions are identical. If they are identical, the ratio inside (3) is the same for every path, so a single sampled path gives the exact likelihood and the inequality becomes an equality. For the model, that limit is infinitely many infinitely small steps with a perfect reverse kernel, which is what Figure 4 approaches as $T$ grows at full training. Small steps alone are not enough; the reverse kernels also have to match. The paper also uses the physics to frame the schedule: in thermodynamics the schedule taken between equilibrium states determines how much free energy is lost, and in the model the schedule $\beta_t$ affects how tight $K$ can get; the paper learns it by gradient ascent on $K$.

Inpainting and denoising: multiplying distributions

Many uses of a generative model are conditional: fill in a missing region of an image, or remove noise from a corrupted one. Both mean multiplying the model by a second function $r(\mathbf{x})$ that encodes the observation, and renormalizing:

For inpainting, $r$ is a delta function that pins the known pixels to their observed values and is constant over the missing ones. For denoising, $r$ is the Gaussian likelihood of the noisy image given a candidate original. The renormalization needs a new normalizing constant, the same kind of integral that makes energy models intractable, and the paper notes that multiplying distributions is costly for variational autoencoders, GSNs, NADEs and most graphical models.

The figure shows what (6) produces for the swiss roll when $r$ observes only the horizontal coordinate. A vertical line crosses the spiral several times, so the conditioned distribution has several separate arcs.

Under a diffusion model the product can be built into the sampler. The paper multiplies every intermediate distribution of the reverse chain by $r$, using the same $r$ at every step in its experiments, which changes each reverse step to

The chain also starts from $\pi \cdot r$ instead of $\pi$; in the inpainting figure of the paper, that start is the image with its missing region replaced by Gaussian noise. Each reverse step is a narrow Gaussian. If $r$ is smooth over the width of that Gaussian, a first-order expansion of $\log r$ around the mean (Appendix C) shifts the mean and leaves the covariance alone:

The shift has a plus sign, it is the covariance times the gradient of $\log r$, and the gradient is evaluated at the unshifted mean $\mathbf{f}_\mu$, not at $\mathbf{x}^{(t)}$. Steering a diffusion sampler with the gradient of a log-likelihood at every step is the mechanism Dhariwal and Nichol named classifier guidance in 2021, with $r$ a classifier's probability of the target class and a scale on the gradient; classifier-free guidance (2022) removed the separate classifier.

The approximation (7) holds only when $r$ is smooth relative to $\mathbf{f}_\Sigma$. For a Gaussian observation of width $\sigma_r$, the shift moves the mean a fraction $g = \mathbf{f}_\Sigma / \sigma_r^2$ of the way to the observed value, so the remaining offset is multiplied by $1 - g$ each step. When $g > 2$ that factor has magnitude above 1: the step lands further from the observed value than it started, on the other side. If enough steps do this, the offsets compound and the chain diverges. The exact product of the two Gaussians moves the mean a fraction $\mathbf{f}_\Sigma / (\mathbf{f}_\Sigma + \sigma_r^2)$, which is always below 1. When $r$ can be multiplied with a Gaussian in closed form, the paper multiplies exactly instead, and the reference sampler does so for both demonstrations: for denoising it combines the two Gaussians into one with precision equal to the sum of their precisions, and for inpainting it sets each known pixel to its value with zero variance at every step.

The figure runs the conditioned 40-step chain both ways, with the same noise draws. At the default width $\sigma_r = 0.17$, $g$ exceeds 2 on 22 of the 40 steps, the product of $|1 - g|$ over the chain is about $3 \times 10^{7}$, and the first-order samples are thrown off the plot. The exact product puts about 90% of the same samples within $2\sigma_r$ of the band's center, on the arcs that Figure 5 lights up. At $\sigma_r = 0.41$ the first-order version also works: the first five, noisiest steps have $g$ up to 5.9 and overshoot, but the remaining steps have $g < 2$ and pull the samples back. Push the slider to $\sigma_r = 0.02$ with the exact product and some samples stay in the band but sit between the two arcs, off the roll. Multiplying each step by $r$ and renormalizing step by step is itself an approximation: the paper replaces the exact modified step, whose normalizer depends on $\mathbf{x}^{(t)}$, with a normalized one, and notes the two agree when $r$ is a small perturbation of each step.

The paper's inpainting result uses the exact case. A diffusion model trained on bark textures ($T = 500$) fills in the central 100×100-pixel region of a bark image by sampling the missing pixels from the posterior given the rest, and a crack entering from the left continues into the filled region. The denoising demonstration corrupts CIFAR-10 test images with Gaussian noise of variance 1 (a signal-to-noise ratio of 1) and samples clean images from the posterior, with $r$ the Gaussian noise likelihood.

Results: what the 2015 numbers mean

The paper reports the bound $K$ on held-out data for every dataset, and also $K - L_{\text{null}}$, the improvement over an isotropic Gaussian (or independent binomial) model of the same data. Higher is better for both.

• Swiss roll: $K = 2.35$ bits, 6.45 bits above the Gaussian baseline.
• Binary heartbeat (length-20 sequences with a 1 in every fifth bin): $K = -2.414$ bits per sequence. The true distribution has $\log_2(1/5) = -2.322$, so the model is within 0.092 bits of perfect.
• Dead leaves (synthetic images of overlapping discs that mimic natural-image statistics): 1.489 bits per pixel, against 1.244 for MCGSM, the previous best, so the paper claims state of the art. Since 1.489 is a lower bound, the model's true log-likelihood is at least that.
• CIFAR-10: $K = 5.4 \pm 0.2$ bits per pixel, 11.5 above the Gaussian baseline, on data dequantized with uniform noise. An earlier arXiv version reported higher numbers that came from the model learning the 8-bit quantization of the pixels; this is the corrected figure.
• Bark: $K = -0.55$ bits per pixel, 1.5 above the baseline.

MNIST is reported differently. Most earlier MNIST likelihoods were Parzen-window estimates, which fit a kernel density to model samples and score test digits under it, so the paper does the same with the code from Goodfellow et al. (2014). The diffusion model scores 317 ± 2.7 bits, against 325 ± 2.9 for the adversarial net, 309 ± 1.6 for a deep GSN, 199 ± 2.9 for a DBN, 174 ± 2.3 for a stacked contractive autoencoder, and 349 ± 3.3 for the training data itself. The paper calls its result comparable to these methods, and it trails the GAN.

No single row is the main result. The paper's claim is that one method, trained the same way, fits a 2-D toy distribution, binary sequences, digits, natural images and textures, and for all of them can draw samples, report a likelihood bound, and compute posteriors such as an inpainting.

What later papers changed

DDPM (Ho, Jain and Abbeel, 2020) kept the forward process and the bound but changed the parameterization: the network predicts the noise $\boldsymbol{\epsilon}$ that was added, the reverse variance is fixed instead of learned, and the weighted sum of KL terms is replaced by an unweighted squared error on $\boldsymbol{\epsilon}$. That version produced sharp images. A year earlier, NCSN (Song and Ermon, 2019) learned the score $\nabla_{\mathbf{x}} \log p$ at several noise levels and sampled with Langevin dynamics, and Score-SDE (2021) showed that both are discretizations of one stochastic differential equation. The continuous-time version of Feller's reverse-process result that Score-SDE builds on is Anderson (1982), which this paper does not cite.

Later work moved further from the 2015 chain: DDIM sampled in tens of steps instead of a thousand, latent diffusion ran the chain in an autoencoder's latent space, and flow matching replaced the chain with a learned velocity field. The guidance used in text-to-image models descends from the multiplying-distributions step in (7).