DDIM: Denoising Diffusion Implicit Models

Get 1,000-step diffusion quality from 20 to 100 steps, with the same trained network.

DDIM changes only the sampling loop of a diffusion model. It defines a family of samplers that all fit a network trained as a standard diffusion model (DDPM), and the deterministic member of that family can skip most of the 1,000 noise levels.

Because the DDIM sampler is deterministic, the starting noise works as a latent code: the same code gives the same image at any step count, two codes can be blended, and a real image can be encoded and reconstructed.

Why DDPM sampling is slow

A denoising diffusion probabilistic model (DDPM) generates an image by starting from pure Gaussian noise and removing a little of it at a time. A standard DDPM uses $T = 1000$ noise levels, and every level costs one forward pass of a large U-Net, the image-to-image network that predicts the noise. The passes cannot run in parallel, because each one needs the output of the one before. The DDIM paper measures the cost on one Nvidia 2080 Ti: about 20 hours to sample 50,000 images of $32\times32$ pixels, against under a minute for a GAN, and nearly 1,000 hours for 50,000 images at $256\times256$.

Taking 50 large steps instead of 1,000 small ones would cut the cost 20×, but a DDPM sampler resists this because it was derived as the reverse of one particular noising process: a Markov chain in which each step adds a small amount of Gaussian noise to the previous step's output. The reverse of that chain is a 1,000-step procedure, and the DDPM derivation gives no reason why a 50-step version should work. DDIM's framework does justify skipping, for DDPM's sampler as well, but the results section shows the DDPM sampler still degrades fast when the steps are few.

DDIM keeps the trained network and derives a new sampler. The argument goes through four ideas, one section each after a short recap: the training loss only ever looks at one noise level at a time; so many different noising processes are compatible with the same trained network; one of them gives a deterministic sampler; and a deterministic sampler can take large steps and run backwards as an encoder.

What the trained network computes

DDIM reuses DDPM's training unchanged, so the notation comes first. The forward process turns an image $\mathbf{x}_0$ into noise over $T$ steps. For a 32×32 RGB image, $\mathbf{x}_0$ is a tensor of shape 3 × 32 × 32, 3,072 numbers scaled to $[-1, 1]$. The forward process is built so that you can jump straight to any noise level $t$ in closed form:

$\alpha_t$ is a number between 0 and 1 that falls as $t$ grows. It sets how much of the image survives: $\sqrt{\alpha_t}$ scales the image down and $\sqrt{1-\alpha_t}$ scales the noise up, and because $\alpha_t + (1-\alpha_t) = 1$ a unit-variance image stays at unit variance. With the CIFAR-10 schedule (per-step noise rising linearly from $10^{-4}$ to 0.02), $\alpha_{100} = 0.897$, $\alpha_{300} = 0.396$, $\alpha_{500} = 0.079$ and $\alpha_{1000} = 4.0\times10^{-5}$. By convention $\alpha_0 = 1$, no noise at all.

The paper's $\alpha_t$ is cumulative. It is the product of all the per-step signal factors up to step $t$, the quantity the DDPM paper and most tutorials call $\bar\alpha_t$ (alpha-bar). DDPM's per-step $\alpha_t = 1-\beta_t$ appears here as the ratio $\alpha_t/\alpha_{t-1}$. The paper states this only in Appendix C.2, so formulas copied from a DDPM tutorial will look wrong next to it until the two notations are lined up. This page uses the cumulative $\alpha_t$ throughout.

The network $\boldsymbol{\epsilon}_\theta^{(t)}(\mathbf{x}_t)$ takes a noisy image and its noise level and returns a tensor of the same shape: its estimate of the noise $\boldsymbol{\epsilon}$ that was mixed in. Training picks a training image, a level $t$ and a noise draw, builds $\mathbf{x}_t$ with (1), and penalizes the squared error:

$\boldsymbol{\gamma}$ is a vector of $T$ positive weights, one per noise level. DDPM trains with every weight equal to 1, which the paper writes $L_{\mathbf{1}}$. The subscript is the all-ones vector; the loss is a squared L2 error, not an L1 loss. The same $L_{\mathbf{1}}$ objective trains the noise-conditional score networks of Song and Ermon, where it appears as denoising score matching, which is why DDPMs and score-based models can share one network.

Predicting the noise is equivalent to predicting the image, because (1) can be solved for $\mathbf{x}_0$. The network's guess of $\mathbf{x}_0$ is

Take one pixel. Say $\alpha_t = 0.30$, the noisy pixel is $x_t = 1.0$, and the network predicts $\hat\epsilon = 0.5$ for it. Then (3) gives

The loss only sees one noise level at a time

Look at what each term of (2) uses: one training image, one noise level, one noise draw. It builds $\mathbf{x}_t$ directly from $\mathbf{x}_0$ with (1) and scores the network on undoing that single jump. No term involves $\mathbf{x}_t$ and $\mathbf{x}_{t-1}$ together, so the loss says nothing about how the noisy versions of one image at neighbouring levels are related. In probability terms, the loss depends only on the marginals $q(\mathbf{x}_t \mid \mathbf{x}_0)$, the distribution of the noisy image at each level on its own, and not on the joint distribution of the whole sequence $\mathbf{x}_1, \dots, \mathbf{x}_T$.

An exam graded question by question against an answer key cannot tell in what order the student answered. In the same way, any noising process whose marginal at every level is (1) produces exactly the same loss, so the network trained for DDPM's process is also the trained network for any of them.

DDPM's Markov chain draws fresh noise at every step: $\mathbf{x}_t$ is a scaled-down $\mathbf{x}_{t-1}$ plus a small new Gaussian, a random walk whose per-step noise is sized so that its spread matches (1) at every level. A second process draws a single noise tensor $\boldsymbol{\epsilon}$ once and builds every level from (1) with that same $\boldsymbol{\epsilon}$. Each $\mathbf{x}_t$ on its own has distribution (1), so the marginals match, but the sequence is a smooth curve fixed by $\boldsymbol{\epsilon}$, where DDPM's is a jagged walk. The second process is the $\sigma = 0$ member of the family in the next section, the one behind DDIM. To reverse it, given $\mathbf{x}_t$ and $\mathbf{x}_0$, recover $\boldsymbol{\epsilon}$ from (1) and put it back at the lower level, with no new random draw.

A family of forward processes

The paper connects these two extremes with a family of processes indexed by a vector $\boldsymbol{\sigma} = (\sigma_1, \dots, \sigma_T)$ of non-negative numbers. Each member is specified by its reverse conditional, which says where $\mathbf{x}_{t-1}$ is when both $\mathbf{x}_t$ and $\mathbf{x}_0$ are known:

Read the mean in two parts. $\sqrt{\alpha_{t-1}}\,\mathbf{x}_0$ is $\mathbf{x}_0$ scaled for level $t-1$. The fraction $(\mathbf{x}_t - \sqrt{\alpha_t}\mathbf{x}_0)/\sqrt{1-\alpha_t}$ is the unit noise that is in $\mathbf{x}_t$, recovered by solving (1); the mean reuses it with weight $\sqrt{1-\alpha_{t-1}-\sigma_t^2}$. On top of the mean, the step adds fresh Gaussian noise with variance $\sigma_t^2$.

The weight is chosen so that the noise variances add up to what (1) requires at level $t-1$. The reused noise has unit variance, so it contributes $1-\alpha_{t-1}-\sigma_t^2$; the fresh noise contributes $\sigma_t^2$; and independent variances add:

The sum is the noise variance of the marginal (1) at level $t-1$, whatever $\sigma_t$ is. The paper proves the general statement (Lemma 1): every $\boldsymbol{\sigma}$ gives a process with marginals exactly (1) at every level. The square root needs $\sigma_t^2 \le 1-\alpha_{t-1}$, a condition the paper leaves implicit. At $\sigma_t = 0$ the step adds nothing new and $\mathbf{x}_{t-1}$ is a fixed function of $\mathbf{x}_t$ and $\mathbf{x}_0$, the reuse-one-noise process from the previous section. At the particular $\sigma_t$ given in (6) below, the family reproduces DDPM's Markov chain.

Figure 1 draws sample paths from the family for one clean pixel. The amber band is the marginal (1), which the family is built to keep fixed. Drag $\eta$, a single scale on all the $\sigma_t$ that (6) makes precise, and watch the paths change while the band stays put:

The paths at different $\eta$ are different processes, so a natural worry is that each needs its own trained network. The paper's Theorem 1 answers it. Each member of the family has its own variational training objective $J_{\boldsymbol{\sigma}}$ (the evidence lower bound a model would be trained on if that member were its forward process), and for every $\boldsymbol{\sigma} > \mathbf{0}$ there exist positive weights $\boldsymbol{\gamma}$ and a constant $C$ with

The weights $\boldsymbol{\gamma}$ depend on $\boldsymbol{\sigma}$, so this is a reweighted version of (2), not $L_{\mathbf{1}}$ itself. The step to "the DDPM network works for every σ" uses one more fact: if the network had separate weights for each noise level, each term of (2) could be minimized on its own, and the minimizer would not depend on $\boldsymbol{\gamma}$. A real network shares one set of weights across all levels, so that step is an idealization. The experiments show that the $L_{\mathbf{1}}$-trained network samples well across the family.

How one DDIM sampling step works

At sampling time $\mathbf{x}_0$ is unknown. The sampler substitutes the network's guess $\hat{\mathbf{x}}_0$ from (3) into the reverse conditional (4), and the recovered unit noise becomes the network's prediction $\boldsymbol{\epsilon}_\theta$. That gives one update rule for every member of the family:

The labels are the paper's. The first term places the predicted image at the next level's scale. The second adds back the predicted noise, scaled for the next level; it points from the guess toward $\mathbf{x}_t$, and it is deterministic, because $\boldsymbol{\epsilon}_\theta^{(t)}(\mathbf{x}_t)$ is a fixed output of the network. Only the third term, a fresh draw $\boldsymbol{\epsilon}_t \sim \mathcal{N}(\mathbf{0},\mathbf{I})$ times $\sigma_t$, is random.

Set $\sigma_t = 0$ at every step and nothing random is left after the initial draw of $\mathbf{x}_T$. The paper calls the resulting model the denoising diffusion implicit model. "Implicit" is used in the sense of an implicit probabilistic model, one that produces samples by pushing a latent variable (here $\mathbf{x}_T$) through a fixed procedure (here the sampling loop), as a GAN does, instead of defining a density step by step.

Continue the one-pixel example, stepping from $\alpha_t = 0.30$ to $\alpha_{t-1} = 0.45$. The noise budget at the new level is $1-\alpha_{t-1} = 0.55$. With $\sigma_t = 0$ all of it goes to the reused noise:

This puts the pixel exactly where it would sit at level $t-1$ if $\hat x_0 = 1.062$ were the true image and $0.5$ the true noise. With the DDPM value of $\sigma_t$ from (6) below, $\sigma_t = 0.512$ and the direction coefficient drops to $\sqrt{0.55 - 0.262} = 0.537$, so the mean is $0.712 + 0.537\cdot 0.5 = 0.981$ and the pixel lands at $0.981 + 0.512\,z$ for a fresh standard normal $z$. Figure 2 does the same split in two dimensions, with an exact denoiser for a four-cluster toy dataset:

To compare deterministic and random sampling with one knob, the paper scales a fixed shape of $\boldsymbol{\sigma}$ by a scalar $\eta \ge 0$:

$\eta = 0$ is DDIM. $\eta = 1$ is DDPM: with this $\boldsymbol{\sigma}$ the forward process becomes Markovian again and (5) is DDPM's own sampler. Values in between are valid samplers that use the same network. In the worked step, $\eta$ can go up to 1.449 before $\sigma_t^2$ uses the whole budget and the direction term vanishes; that is the right end of the slider in Figure 2.

The paper also tests a third setting, $\hat\sigma_t = \sqrt{1-\alpha_t/\alpha_{t-1}}$, the larger variance used in Ho et al.'s code for their CIFAR-10 samples. It is not a point on the $\eta$ line. Appendix D.3 gives its update: the same first two terms as $\eta = 1$, with noise $\hat\sigma_t\boldsymbol{\epsilon}_t$ in place of $\sigma_t(1)\,\boldsymbol{\epsilon}_t$. In the worked step $\hat\sigma_t = 0.577$, so the total noise variance is $0.288 + 0.333 = 0.621$, 13% more than the 0.55 that (1) calls for. Between adjacent levels of the 1,000-step CIFAR-10 schedule the excess is at most $10^{-4}$; across a large skip it is large, and the paper attributes $\hat\sigma$'s poor few-step results to this extra noise.

Skipping steps

The marginals argument also allows skipping. The derivation of (4) and (5) never used the fact that $t-1$ is adjacent to $t$; it only needs the marginal (1) at the two levels involved. So pick an increasing subsequence $\tau = (\tau_1, \dots, \tau_S)$ of $1, \dots, T$, define the process on those levels alone, and sample along $\tau$ in reverse, replacing $t$ and $t-1$ in (5) with $\tau_i$ and $\tau_{i-1}$. The cost is $S$ network calls instead of $T$, and the network is the one trained on all 1,000 levels.

The paper tries two spacings, each of the form $\tau_i = \lfloor c\,i\rfloor$ (linear) or $\tau_i = \lfloor c\,i^2\rfloor$ (quadratic), with $c$ set so the last level is close to $T$. Quadratic spacing puts more steps at low noise, where fine detail is resolved. It used quadratic for CIFAR-10 and linear for the other datasets, reporting that each gave slightly better FID than the alternative on its dataset. Figure 3 plots both against the CIFAR-10 schedule's noise variance:

Under this schedule the noise variance passes 0.92 by $t = 500$, so with linear spacing about half of the visited levels sit where the image is almost all noise. At $S = 20$, linear spacing puts 4 levels at $t \le 200$ and quadratic puts 8.

Whether skipping works well depends on $\eta$. A large jump makes $\hat{\mathbf{x}}_0$ a cruder estimate, because the network has to guess $\mathbf{x}_0$ from a much noisier input. The deterministic update then carries that estimate's own predicted noise forward, while the stochastic one replaces part of it with fresh noise that the remaining few steps must remove. The full sampler is one loop for every setting:

# One sampling run. Same trained eps_theta for every setting;
# only tau (which levels) and eta (how much fresh noise) change.
def sample(eps_theta, alpha, tau, eta):  # alpha[t]: cumulative, alpha[0] = 1
    x = randn(3, 32, 32)                 # x_T: pure Gaussian noise
    for i in reversed(range(len(tau))):  # walk reversed(tau): noise -> data
        t = tau[i]                       # current level
        s = tau[i - 1] if i > 0 else 0   # next, less noisy level
        a_t, a_s = alpha[t], alpha[s]
        eps = eps_theta(x, t)                          # one network call
        x0 = (x - sqrt(1 - a_t) * eps) / sqrt(a_t)     # eq (3)
        sig = eta * sqrt((1 - a_s) / (1 - a_t)) * sqrt(1 - a_t / a_s)  # eq (6)
        dir = sqrt(1 - a_s - sig**2) * eps             # deterministic
        x = sqrt(a_s) * x0 + dir + sig * randn(3, 32, 32)  # eq (5)
    return x                                           # a sample x_0

Each pass through the loop is one network call, so $S = 50$ costs 50 forward passes. With eta = 0 the randn inside the loop is multiplied by zero and the only randomness is the first line.

How much faster, at what quality

The paper trains one model per dataset at $T = 1000$ with $L_{\mathbf{1}}$ and changes only $\tau$ and $\eta$. Quality is measured with FID (Fréchet Inception Distance), which compares statistics of generated and real images in the feature space of an image classifier; lower is better. Figure 4 plots the paper's Table 1 for DDIM, DDPM and $\hat\sigma$:

On CIFAR-10 at 10 steps, DDIM scores 13.36, DDPM 41.07 and $\hat\sigma$ 367.43. At 100 steps DDIM reaches 4.16, close to its own 1,000-step 4.04. On CelebA the 100-step DDPM (13.93) matches the 20-step DDIM (13.73), so DDIM gets there with a fifth of the network calls. The two intermediate settings in Table 1 fall in between at every step count: at 10 steps on CIFAR-10, $\eta = 0.2$ gives 14.04 and $\eta = 0.5$ gives 16.66.

The paper's headline is that DDIM matches 1,000-step quality in 20 to 100 steps, which it states as a 10× to 50× speedup in wall-clock time. Sampling time is linear in the number of steps, so a 50-step run takes about 1/20 of the 1,000-step time; the 20 hours for 50,000 CIFAR-10 images become roughly one. The paper also writes "10× to 100×" in two places, as the range of step reductions it tests (1,000 steps down to 100 or 10), not as a matched-quality figure.

At the full 1,000 steps the order flips: $\hat\sigma$ scores 3.17 on CIFAR-10 against DDIM's 4.04, and 3.26 against 3.51 on CelebA. With every step taken, the added noise helps a little; DDIM's advantage is confined to small step counts, the regime that sets sampling cost.

The starting noise as a latent code

With $\sigma_t = 0$ the sample is a deterministic function of $\mathbf{x}_T$. The paper finds that this function barely depends on the step count: for a fixed $\mathbf{x}_T$, images generated with 10, 20, 50, 100 and 1,000 steps share their high-level features, such as pose, layout and colors, and the 20-step image is often already very close to the 1,000-step one.

Figure 5 shows the same effect in a toy. Eight fixed starting points are each decoded by the exact denoiser of an eight-cluster dataset. A ring marks where each lands with a 100-step run, a dot where it lands with $S$ steps. Under DDIM the dots sit on their rings from about 4 steps up. Switch to DDPM and the dots move off their rings at every $S$, including 100, because the reference run draws its own fresh noise:

Since $\mathbf{x}_T$ determines the image, it works like the latent code of a GAN or a variational autoencoder. The paper interpolates in it: draw two codes $\mathbf{x}_T^{(0)}$ and $\mathbf{x}_T^{(1)}$, blend them, decode each blend with 50 DDIM steps, and the images change smoothly from one sample to the other. The blend is spherical linear interpolation (slerp), not a straight average:

The paper does not say why it chose slerp, but the geometry of high-dimensional Gaussians gives a reason. A 3,072-dimensional standard normal vector almost always has length close to $\sqrt{3072} \approx 55.4$, give or take about 2. Two independent codes are nearly orthogonal, so their straight-line midpoint has length about $55.4/\sqrt 2 \approx 39.2$, a length the network never saw at $t = T$. Slerp moves along the arc between the codes and keeps the length near 55.4. Figure 6 plots the length along both blends:

A DDPM cannot do this with one code, because its output also depends on the fresh noise drawn at every step: the same $\mathbf{x}_T$ gives a different image on each run. The paper notes that interpolating all $T$ noise draws might work instead. Running the deterministic sampler backwards gives the remaining latent-code operation, finding the code of a real image, covered in the next section.

DDIM as an ODE solver

The deterministic update can be rewritten as one step of Euler's method, the simplest numerical method for an ordinary differential equation (ODE): from the current point, move a step along the current velocity. Change variables to $\bar{\mathbf{x}} = \mathbf{x}/\sqrt{\alpha}$ and $\sigma = \sqrt{1-\alpha}/\sqrt{\alpha}$. By (1), $\bar{\mathbf{x}}_t = \mathbf{x}_0 + \sigma_t\,\boldsymbol{\epsilon}$: the clean image plus $\sigma_t$ times unit noise, with $\sigma_t$ growing from 0 at the clean end to 157 at $t = 1000$ on the CIFAR-10 schedule. (This $\sigma_t$ is a different quantity from the sampler's noise scale in (4) to (6); the overloaded symbol is the paper's.) The DDIM update (5) with $\sigma_t = 0$ becomes

an Euler step with step size $\sigma_{t-1}-\sigma_t$ for the ODE

The network's noise prediction is the velocity and $\sigma$ plays the role of time. Since $\sigma^2 + 1 = 1/\alpha$, the network input $\bar{\mathbf{x}}/\sqrt{\sigma^2+1}$ is just $\mathbf{x}_t$, the kind of noisy image it was trained on. Check it on the one-pixel example, where $\sigma_t = \sqrt{0.70/0.30} = 1.528$, $\sigma_{t-1} = \sqrt{0.55/0.45} = 1.106$ and $\bar x_t = 1.0/\sqrt{0.30} = 1.826$. The Euler step lands on the DDIM result from before:

An ODE can be integrated in either direction. Run the same update from the clean end toward the noisy end and it encodes an image into its $\mathbf{x}_T$; run it back and it decodes. Each direction makes discretization error, so the round trip lands near the start, not on it, and the gap shrinks as steps are added. Figure 7 runs both directions on a one-dimensional toy with three data clusters, where the denoiser is exact and only step size causes error:

In the default view (6 steps, $x_0 = 1.30$), the code comes out at 0.77 instead of the 1,000-step value 1.04, and decoding it lands at 0.80, between two clusters and 0.50 away from the start. At 20 steps the miss is 0.012, and at 100 steps 0.004. The paper measures the same thing on the CIFAR-10 test set, encoding and decoding with the same $S$ and reporting per-dimension squared error with pixels scaled to $[0, 1]$:

An error of 0.0009 is a root-mean-square pixel error of 0.03, about 8 levels out of 255. A DDPM has no such round trip, because decoding draws new noise at every step. The encoder is the sampling loop run forwards:

# Encoding: the same update with eta = 0, run from clean to noisy.
def encode(eps_theta, alpha, tau, x):    # x: a real image x_0
    levels = [0] + list(tau)             # 0 -> tau_1 -> ... -> tau_S
    for t, s in zip(levels[:-1], levels[1:]):   # s > t: noisier
        eps = eps_theta(x, t)
        x0 = (x - sqrt(1 - alpha[t]) * eps) / sqrt(alpha[t])
        x = sqrt(alpha[s]) * x0 + sqrt(1 - alpha[s]) * eps
    return x                             # x_T, the image's code

Equation (7) also connects DDIM to score-based SDEs, which describe diffusion in continuous time and come with a deterministic "probability flow" ODE. Proposition 1 of the paper shows that, with the optimal network, (7) is the probability flow ODE of the variance-exploding SDE, the continuous-time form of the NCSN noise process. That ODE is usually written in $\mathrm{d}t$ with a factor $\tfrac12 g(t)^2$; (7) is written in $\mathrm{d}\sigma$ with no ½, and the change of variables $\mathrm{d}\sigma^2/\mathrm{d}t = 2\sigma\,\mathrm{d}\sigma/\mathrm{d}t$ absorbs the ½. The two samplers still differ: DDIM takes Euler steps in $\sigma$, the score-SDE sampler takes them in $t$, and with few steps those land in different places. The view of sampling as solving an ODE with a learned velocity also links DDIM to Neural ODEs, which Section 5.4 cites alongside normalizing flows when describing the round trip.