Diffusion Models Beat GANs on Image Synthesis

A better U-Net and a classifier's gradient let a diffusion model beat BigGAN on ImageNet.

Dhariwal and Nichol tune the diffusion model's denoising network, then add classifier guidance: during sampling, each denoising step is shifted along the gradient of a classifier, and one scale parameter sets how far, trading sample variety for sample quality.

In 2021 the best ImageNet samples came from GANs, which get them partly by leaving out parts of the data. This paper reaches a lower FID with a diffusion model that leaves out less.

GANs and diffusion models in 2021

Before this paper, the strongest class-conditional ImageNet generators were GANs (generative adversarial networks), in which a generator learns to produce images that a second network, the discriminator, cannot tell apart from real ones. BigGAN-deep was the state of the art at 256×256 with an FID of $6.95$. FID is the standard sample-quality score, defined in a later section; lower is better. The paper names two weaknesses of GANs: they capture less of the data's diversity than the best likelihood-based models, and they are "often difficult to train, collapsing without carefully selected hyperparameters and regularizers".

Diffusion models are likelihood-based. They train with a plain regression loss and cover the data distribution well, but in 2021 their ImageNet samples still scored worse than a good GAN's. Prafulla Dhariwal and Alex Nichol, at OpenAI, hypothesize that the gap comes from at least two factors. GAN architectures had been explored and refined for years, while diffusion models still used the network from one 2020 paper. And a GAN can trade diversity for fidelity: BigGAN's truncation trick draws its input noise from a normal distribution cut off near the centre, which gives cleaner images from fewer modes of the data, and diffusion models had no equivalent control.

The paper answers each factor with one contribution. An architecture sweep produces a stronger denoising network, which the authors call ADM (ablated diffusion model). Classifier guidance adds the missing control: during sampling, the gradient of an image classifier shifts each denoising step toward a chosen class, and a scalar $s$ sets how hard. Guided ADM reaches FID $2.97$ on ImageNet 128×128, $4.59$ on 256×256 and $7.72$ on 512×512, against BigGAN-deep's $6.02$, $6.95$ and $8.43$. With only 25 sampling steps it scores $5.98$, $5.44$ and $8.41$, still level with or better than BigGAN-deep at each resolution.

The page follows the paper's order: the diffusion model, reduced to the five equations guidance needs; the architecture changes; the metrics that measure fidelity and diversity separately; the derivation of classifier guidance and the one approximation in it; what the scale does, shown on a toy sampler you can run; the version for the fast DDIM sampler; and the measured results.

A diffusion model in five equations

The full construction is in the DDPM explainer; this section keeps the parts classifier guidance uses. A diffusion model generates images by reversing a fixed noising process. The forward process takes a clean image $x_0$ and mixes in Gaussian noise over $T = 1000$ steps until only noise is left. In the variance-preserving (VP) convention this paper uses, any noise level $t$ can be reached in one jump:

In (1), $\epsilon \sim \mathcal{N}(0,\mathbf{I})$ is the noise and $\bar\alpha_t$ falls from near 1 at the first step to near 0 at the last. On ADM's linear schedule $\bar\alpha_t$ is $0.9999$ at $t=0$, $0.079$ halfway ($t=499$) and $4\times10^{-5}$ at $t=999$. Halfway, the image is scaled by $\sqrt{0.079} = 0.28$ and noise with standard deviation $0.96$ is added. The signal variance $\bar\alpha_t$ and the noise variance $1-\bar\alpha_t$ sum to one, so data scaled to unit variance keeps unit variance at every $t$. (Variance-exploding papers leave the signal unscaled and grow only the noise; the formulas below are for the VP form.)

For a 256×256 RGB image, $x_0$, $\epsilon$ and $x_t$ are tensors of shape 3 × 256 × 256, with pixel values scaled to $[-1, 1]$. Following Ho et al., a network $\epsilon_\theta(x_t, t)$ takes the noised image and the timestep and predicts the noise that was added, trained with a squared error:

Each training step picks an image, a $t$ and an $\epsilon$, builds $x_t$ with (1), and regresses the prediction onto the $\epsilon$ it used. Sampling runs the chain backward. Each reverse step is a Gaussian $p_\theta(x_{t-1} \mid x_t) = \mathcal{N}(\mu_\theta, \Sigma_\theta)$ whose mean is computed from the noise prediction, with $\beta_t = 1-\alpha_t$ the forward process's per-step noise variance:

From Improved DDPM, the authors' previous paper, ADM also learns the variance. The network outputs a vector $v$, and the variance interpolates in log space between two bounds: $\beta_t$ and the forward process's posterior variance $\tilde\beta_t$. It is trained with a hybrid loss, $L_{\text{simple}}$ plus a small weight on the variational bound.

The two bounds differ by orders of magnitude near $t = 0$, which is why the blend is done in log space. A fixed variance works when the number of sampling steps is large; learning it lets the sampler take fewer steps "without much drop in sample quality". ADM's ImageNet samples use 250 steps by default. Below 50 steps the paper switches to the deterministic DDIM sampler, and all its 25-step results use DDIM.

The last equation connects the noise predictor to the score, the gradient of the log-density with respect to the image. A classifier gradient will enter the sampler through it. Given $x_0$, the noised image in (1) is Gaussian around $\sqrt{\bar\alpha_t}\,x_0$ with variance $1-\bar\alpha_t$, so its score is $-(x_t - \sqrt{\bar\alpha_t}\,x_0)/(1-\bar\alpha_t)$. The numerator equals $\sqrt{1-\bar\alpha_t}\,\epsilon$, which leaves $-\epsilon/\sqrt{1-\bar\alpha_t}$. The network does not know $x_0$; its prediction estimates the average $\epsilon$ over the images that could have produced $x_t$, and the same average turns this per-image score into the score of the noised data distribution:

The divisor is the standard deviation $\sqrt{1-\bar\alpha_t}$, not the variance, because one power cancels against the noise in the numerator. The minus sign says density increases in the direction opposite the predicted noise, which is toward the data. The score-matching line of work rests on the same equivalence, and Song et al.'s SDE paper states it in the form used here.

How ADM improves the U-Net

Ho et al.'s denoiser is a U-Net: residual blocks that downsample the image through coarser resolutions and upsample back, with skip connections between matching resolutions. It has one global self-attention layer, at 16×16, with a single head, and adds a projection of the timestep embedding into each residual block. Dhariwal and Nichol test five changes, each applied alone to a baseline with 160 channels and 2 residual blocks per resolution, on ImageNet 128×128 with batch size 256 and 250 sampling steps. FID is measured after 700K and 1200K training iterations; the baseline scores $15.33$ and $13.21$.

• Depth: 4 residual blocks per resolution at 128 channels, a deeper and narrower network of similar size.
• Heads: 4 attention heads instead of 1. In self-attention every spatial position takes a weighted average over all positions; with several heads the channels are split and each slice runs its own attention pattern.
• Attention resolutions: self-attention at 32×32, 16×16 and 8×8 instead of 16×16 only.
• BigGAN up/downsampling: the residual block from BigGAN for every change of resolution, following Song et al.
• Rescaled residuals: every residual connection multiplied by $1/\sqrt{2}$.

Figure 1 plots each change's FID difference from the baseline; bars to the left of the zero line are improvements.

Rescaling residuals raises FID by $0.16$ at 700K and $0.25$ at 1200K, and was dropped. Depth lowers FID ($-0.21$ and $-0.48$) but takes longer in wall-clock time to reach the same FID as a wider model, so it was dropped too. The other three combined give $-3.14$ at 700K, more than the $-2.46$ sum of their separate gains ($-0.54$, $-0.72$, $-1.20$); the paper describes the changes as having "a positive compounding effect".

A second sweep (Table 2, 128 base channels, 700K iterations, baseline FID $14.08$ with one head) sizes the attention two ways. Fixing the number of heads at 2, 4 or 8 lowers FID by $0.50$, $0.97$ and $1.17$. Fixing the channels per head at 32, 64 or 128 gives $-1.36$, $-1.03$ and $-1.08$. At a given width, fewer channels per head means more heads, so both sweeps vary the same quantity. 32 channels per head has the best FID, but 64 is fastest in wall-clock time and matches how a Transformer sizes its heads, so ADM uses 64.

Injecting the timestep and class with AdaGN

Every residual block needs the noise level $t$ and, in a class-conditional model, the class $y$. Each becomes an embedding vector and the two are summed, $e = e_t + e_{\text{class}}$ (an unconditional model has only $e_t$). Ho et al. projected the embedding and added it to the block's activations. ADM uses adaptive group normalization (AdaGN). GroupNorm splits a block's channels into groups and normalizes each group to zero mean and unit variance using the activations' own statistics. AdaGN then scales and shifts each channel by amounts computed from the embedding:

In (6), $h$ is the block's activation after its first convolution, and for a block with $C$ channels the linear layer outputs $2C$ numbers: a scale and a shift per channel. Only GroupNorm reads statistics from $h$; the scale and shift depend only on the timestep and class. The official code computes out_norm(h) * (1 + scale) + shift, so a zero output means no rescaling. FiLM and StyleGAN's adaptive instance norm use the same scale-and-shift. Adding a vector to a normalized channel moves its mean; multiplying also changes its spread, so a block can amplify or damp a channel depending on the noise level. Toggle Figure 2 between the two and compare the widths of the coloured curves against the grey reference.

Both models in that ablation use 128 channels, multi-resolution attention with 64 channels per head and BigGAN up/downsampling, trained for 700K iterations. The final default architecture has 2 residual blocks per resolution, a width that grows at coarser resolutions, 64 channels per head, attention at 32, 16 and 8, BigGAN up/downsampling, and AdaGN. The ImageNet 256×256 model built this way has 256 base channels, channel multipliers 1, 1, 2, 2, 4, 4 across six resolutions, and 554M parameters.

The architecture alone sets the state of the art on the three LSUN datasets (bedrooms, horses, cats) and on ImageNet 64×64, where ADM with dropout reaches FID $2.07$ against BigGAN-deep's $4.06$. On ImageNet 128×128 unguided ADM scores $5.91$, slightly better than BigGAN-deep's $6.02$ but behind LOGAN's $3.36$. At 256 and 512 it is far behind: $10.94$ against $6.95$, and $23.24$ against $8.43$. These are the resolutions where guidance changes the result: guided, the same architecture reaches $4.59$ and $7.72$.

Measuring fidelity and diversity: FID, precision and recall

Guidance raises one property of a set of samples at the expense of another, so the paper needs metrics that measure them separately. All of them work on features from an Inception-V3 network, which compares images by what an ImageNet classifier extracts from them.

FID (Fréchet Inception Distance, Heusel et al.) fits a Gaussian to the features of real images, with mean $\mu_r$ and covariance $\Sigma_r$, and another to the features of generated images ($\mu_g$, $\Sigma_g$), then computes

This is the squared 2-Wasserstein distance between the two Gaussian fits, the least average squared distance that one Gaussian's probability mass must travel to become the other. It compares the fits, not the actual feature distributions, so differences the Gaussians do not capture do not count. FID goes up when samples look wrong ($\mu_g$ moves) and when they lack variety ($\Sigma_g$ shrinks), which is why the paper uses it as its overall score. sFID computes the same quantity on spatial feature maps instead of pooled features, to catch errors in layout. Inception Score (IS) rewards samples that an ImageNet classifier labels confidently and that spread across the classes. It does not reward variety within a class, so a model that memorizes a small subset of the dataset still scores high.

Precision and recall (Kynkäänniemi et al., 2019) measure the two properties separately. Each set of feature vectors defines a manifold estimate: around every point, a sphere that reaches its third-nearest neighbour in the same set, and the manifold is the union of those spheres. Precision is the fraction of generated samples that fall inside the real-data manifold, a measure of fidelity. Recall is the fraction of real images that fall inside the generated-sample manifold, a measure of coverage. The two tests run in opposite directions. A model that puts every sample inside one dense region of the real data scores high precision, while recall drops because most real images have no generated sample near them. Drag the concentration slider in Figure 3 to pull the model's samples onto one mode.

A truncated GAN sits toward the high-precision end: BigGAN-deep at 256×256 has precision $0.87$ and recall $0.28$. Classifier guidance gives a diffusion model a control that moves it along the same axis.

How classifier guidance works: the exact part

GANs for class-conditional synthesis use labels heavily, through class-conditional normalization and discriminators with classifier-like heads. Classifier guidance brings the label in from a separate model: train a classifier $p_\phi(y \mid x_t, t)$ on noised images, and during sampling use its gradient with respect to the image. The paper takes the method from Sohl-Dickstein et al. (2015) and Song et al., and writes $p_\phi(y \mid x_t)$ with the timestep left implicit.

The classifier is the downsampling half of the ADM U-Net, with an attention pool (the pooling layer from CLIP) at the 8×8 resolution producing the output. It is trained on the same noising distribution as the diffusion model, with random crops to reduce overfitting. The classifiers have 43M to 65M parameters, against 296M to 559M for the diffusion models. At each reverse step it takes the current noisy image $x_t$ and the timestep and returns logits over the 1,000 ImageNet classes.

Start from a trained diffusion model whose reverse step $p_\theta(x_t \mid x_{t+1})$ knows nothing about $y$. To condition that step on $y$, it is enough to multiply it by the classifier and renormalize:

with $Z$ a normalizing constant. The appendix proves the fact that (8) relies on: if the forward process is redefined to also carry the label, its noising steps are unchanged, $\hat q(x_{t+1} \mid x_t) = q(x_{t+1} \mid x_t)$, so the unconditional reverse model the network learned is the correct first factor. Both statements are exact. The chain also has to start from $x_T$ conditioned on $y$; the paper notes that a noisy enough forward process makes $x_T$ nearly Gaussian for every class, so every class starts from $\mathcal{N}(0, \mathbf{I})$. On the linear schedule the image's contribution to $x_T$ is scaled by $\sqrt{4\times10^{-5}} = 0.006$. Sampling from (8) exactly is still intractable, because it is a Gaussian multiplied by an arbitrary function computed by a neural network, so the paper approximates it.

The mean shift and its Taylor approximation

Write the model's reverse step as $\mathcal{N}(\mu, \Sigma)$, so its log-density is a quadratic in $x_t$. Over the region that Gaussian covers, replace $\log p_\phi(y \mid x_t)$ by its first-order Taylor expansion around $\mu$: a constant plus $(x_t-\mu)^\top g$, where $g$ is the classifier's log-probability gradient at $\mu$. The log of the product in (8) becomes a quadratic plus a linear term, and completing the square turns it into another Gaussian with the same covariance and a moved mean:

where $g = \nabla_{x_t}\log p_\phi(y \mid x_t)\big|_{x_t=\mu}$. The constant $\tfrac12 g^\top\Sigma g$ left over from completing the square is absorbed into $Z$. The mean moves by $\Sigma g$, the gradient multiplied by the step's covariance. The covariance lies between $\beta_t$ and $\tilde\beta_t$, which on the 250-step schedule are within 8% of each other at the points below, so the same gradient moves the mean by about $0.039\,g$ halfway through sampling and $0.0076\,g$ near the end ($t = 99$). Guidance pushes hard while the image is mostly noise and lightly once it has formed. Algorithm 1 of the paper adds a scale $s$ to the gradient:

In Algorithm 1 and in the official code, the gradient is taken at the current noisy image $x_t$ with the timestep-$t$ classifier, not at the mean $\mu$ as in the derivation: guided-diffusion's condition_mean returns the model mean plus variance * gradient, with the gradient computed from x. One sampling step on a 256×256 image, with the scale applied:

# Algorithm 1: classifier-guided ancestral sampling, gradient scale s
x = randn(3, 256, 256)                  # x_T ~ N(0, I), same start for every class
for t in reversed(range(T)):            # T-1 .. 0
    mu, var = model(x, t)               # unconditional reverse step N(mu, var)
    logp = log_softmax(classifier(x, t))[y]   # noised classifier, at the CURRENT x_t
    g = grad(logp, x)                   # same shape as x
    x = mu + s * var * g + sqrt(var) * randn_like(x)   # var is diagonal
return x                                # x_0, values in [-1, 1]

Figure 4 shows the shift in two dimensions with a two-class classifier. Drag the mean $\mu$ and raise $s$: the arrow is $s\Sigma g$, and the readout gives the target class's probability at $\mu$ and at the shifted mean. Place $\mu$ near the other class, where an unguided sample would be ambiguous, and watch the shift carry the mean back across the boundary.

The linearization needs the classifier's log-probability to be close to a straight line across the width of the step. The paper's condition is low curvature compared with $\Sigma^{-1}$, "reasonable in the limit of infinite diffusion steps, where $\lVert\Sigma\rVert \to 0$". Figure 5 tests it in one dimension, where both sides can be computed. One step $\mathcal{N}(\mu, \Sigma)$ meets a logistic classifier $p(y \mid x) = 1/(1+e^{-3x})$. The amber curve is the exact guided step, the Gaussian times $p(y \mid x)^s$ normalized numerically (the next section shows why a scaled gradient corresponds to this power); the teal curve is the shifted Gaussian from (10).

At the default step ($\mu = -0.6$, step standard deviation $0.5$, $s = 1$) the exact mean is $-0.16$ and the shifted mean $+0.04$. Narrow the step to $0.1$ and they agree to three decimals ($-0.575$ and $-0.574$). Widen it to $1.0$ and the shifted mean lands at $1.97$ against an exact $0.40$: the linearization keeps using the steep gradient it measured at $\mu$, while the real log-probability flattens out once $x$ is inside the class.

Fewer sampling steps make each step wider. With the linear schedule, the reverse-step variance halfway through sampling is $0.010$ with all 1000 steps, $0.039$ with 250 and $0.31$ with 25, standard deviations of $0.10$, $0.20$ and $0.56$. The toy classifier is not in image units, but the ratio carries over: cutting the step count tenfold roughly triples the width of each step, and the linear approximation gets worse. The paper's tuned scales for 25-step DDIM ($1.25$, $2.5$, $9.0$ at 128, 256, 512) are above its 250-step ones ($0.5$, $1.0$, $4.0$).

Why the gradient scale goes above 1

The derivation gives $s = 1$: (8) is the exact conditional, and (10) at $s = 1$ approximates it. On unconditional ImageNet models $s = 1$ did not work. The classifier assigned the intended class about 50% probability on the finished samples, and the samples did not look like that class. Raising the scale brought the classifier's probabilities to nearly 100% and fixed the samples. The paper's corgi figure, from the unconditional 256×256 model, shows the effect: scale $1.0$ gives FID $33.0$, worse than the unguided model's $26.21$, and scale $10.0$ gives FID $12.0$ and recognizable corgis.

A larger scale can be read as a sharper classifier. Multiplying a log-gradient by $s$ gives the gradient of the classifier raised to the power $s$:

Raising $p(y \mid x)$ to a power $s > 1$ amplifies values that are already large relative to small ones, so sampling concentrates on the inputs the classifier is most confident about: higher fidelity, less diversity. The identity holds for any constant $Z$ that does not depend on $x$. A classifier renormalized per input, $p(y \mid x)^s / \sum_{y'} p(y' \mid x)^s$, has a normalizer $Z(x)$ that does depend on $x$, and its gradient would include an extra $-\nabla_x\log Z(x)$ term. The paper says the process "is still theoretically grounded in a re-normalized classifier distribution proportional to $p(y|x)^s$"; read that as a description of what the scale does, a temperature of $1/s$ on the classifier, not as a claim that the sampler draws from a renormalized classifier.

Figure 6 runs guided sampling end to end on a one-dimensional toy where every quantity is exact. Class A is two equal bumps at $-1.6$ and $-0.55$; class B is one bump at $+1.1$ holding half the data. For a mixture of Gaussians the noised density, its score and the noised classifier $p(A \mid x_t)$ all have closed forms, so the only approximations are the paper's: a finite number of steps and the mean shift (10), with the gradient taken at $x_t$. The schedule is ADM's linear one, respaced to the chosen step count. Paths start from $\mathcal{N}(0,1)$ at the top; the histogram shows 3,000 samples at the scrubbed step against the true class-A density at that noise level.

At $s = 0$ the sampler is unconditional and half the samples end in class B. At $s = 1$ with 250 steps, 99.9% end in class A and the bump nearer class B holds 51% of them, against 50% in the true $p(x \mid A)$, so the sampler draws from the correct conditional. At $s = 3$ that bump holds 27% of the class-A samples, and at $s = 10$, 14%. The samples stay in class A, which is what precision measures, while the part of class A that looks most like class B empties, which is what recall measures. With 25 steps at $s = 1$, only 92% of samples land in class A, and with 10 steps 77%: coarse steps under-apply the guidance, and a larger $s$ makes up for it.

Multiplying a classifier's gradient by 10 is also how adversarial examples are made: small changes to an input that drive a classifier to high confidence without changing what a person sees. Guidance at $s = 10$ produces corgis instead. The paper reports that the scale can be raised "by an order of magnitude without obtaining adversarial examples", and separately that the classifier is trained on noised images. Connecting the two is this page's interpretation, not the paper's argument: a classifier that must work on heavily noised inputs cannot rely on the fine, high-frequency patterns that adversarial perturbations exploit, so its gradients point toward coarse, class-level structure. Tsipras et al. (2019) found that adversarially robust classifiers learn features that align better with human perception, which is consistent with this reading.

This $s$ is not the weight $w$ of classifier-free guidance, the follow-up that removes the separate classifier. Classifier-free guidance at weight $w$ corresponds to a classifier-guidance scale of $s = w + 1$, so its $w = 0$ is plain conditional sampling.

Guiding the deterministic DDIM sampler

The 25-step results use DDIM, which maps noise to an image deterministically, with no Gaussian sampled at each step, so (10) has no covariance to shift. The paper instead adapts the score-based route from Song et al. The score of the joint $p_\theta(x_t)\,p_\phi(y \mid x_t)$ is a sum, because the log of a product is the sum of the logs, and (5) supplies the first term:

Define a new noise prediction $\hat\epsilon$ whose score, through (5), is this joint score: multiply (12) by $-\sqrt{1-\bar\alpha_t}$. With the scale on the classifier term,

Adding the classifier gradient to the score subtracts it from the noise, since the score is the noise times $-1/\sqrt{1-\bar\alpha_t}$. DDIM then runs unchanged with $\hat\epsilon$ in place of $\epsilon_\theta$. The paper prints its version of (13), and the matching line of Algorithm 2, without $s$, although Algorithm 2 takes "gradient scale $s$" as an input; the official code multiplies the classifier gradient by classifier_scale before condition_score subtracts it. The DDIM loop, with the scale where the code applies it:

# Algorithm 2: classifier-guided DDIM (deterministic), gradient scale s
x = randn(3, 256, 256)
for t in reversed(range(T)):            # 25 strided steps in the paper's fast runs
    eps = eps_model(x, t)
    g = grad(log_softmax(classifier(x, t))[y], x)
    eps_hat = eps - s * sqrt(1 - abar[t]) * g     # Eq. (13): subtract from the noise
    x0_pred = (x - sqrt(1 - abar[t]) * eps_hat) / sqrt(abar[t])
    x = sqrt(abar_prev[t]) * x0_pred + sqrt(1 - abar_prev[t]) * eps_hat
return x

Results: the trade-off curve and beating BigGAN

On ImageNet 256×256 the paper measures guidance at $s = 0$, 1 and 10 on a conditional and an unconditional diffusion model, both trained for 2M iterations with batch size 256 (Table 4). Figure 7 places those points on the precision/recall plane with BigGAN-deep for reference.

The unguided conditional model has FID $10.94$, precision $0.69$ and recall $0.63$. At $s = 1$: FID $4.59$, precision $0.82$, recall $0.52$, and Inception Score up from $100.98$ to $186.70$. At $s = 10$: precision $0.88$, recall $0.32$, FID $9.11$. FID responds to both fidelity and diversity, so its best value is at an intermediate scale. The unconditional model gets worse at $s = 1$ (FID $33.03$) and reaches $12.00$ at $s = 10$, close to the unguided conditional model's $10.94$; the paper adds that "training directly with the class labels still helps", and guiding the conditional model gives the best result. The headline numbers all come from guided class-conditional models (ADM-G), with scales tuned per setting between $0.5$ (ImageNet 128, 250 steps) and $9.0$ (ImageNet 512, 25-step DDIM).

BigGAN-deep sits at precision $0.87$ and recall $0.28$. At $s = 1$ the guided model gives up a little precision ($0.82$) for much more recall ($0.52$); at $s = 10$ it matches the GAN's precision with recall $0.32$. On ImageNet 128 the paper finds guidance "strictly better than BigGAN-deep when trading off FID for Inception Score", but on precision and recall it is better only up to a precision threshold, beyond which it "cannot achieve better precision".

The headline FIDs (Table 5) use 250 steps: $2.97$ at 128×128, $4.59$ at 256×256 and $7.72$ at 512×512, against BigGAN-deep's $6.02$, $6.95$ and $8.43$. With 25 DDIM steps ADM-G scores $5.98$, $5.44$ and $8.41$, so 25 diffusion-model passes per sample, plus 25 classifier gradients, match BigGAN-deep at every resolution. A two-stage stack, a guided low-resolution model (64×64 for the 256 result, 128×128 for 512) followed by an upsampling diffusion model called ADM-U, reaches $3.94$ at 256 and $3.85$ at 512. Only the low-resolution model is guided. ADM-U on top of an unguided base gives $7.49$ and $9.96$, so guidance and upsampling each contribute.

Diffusion still costs more. In training compute (Table 10, in V100-days), the full ImageNet 256 ADM-G took 916 for the diffusion model plus 46 for the classifier, against an estimated 128 to 256 for BigGAN-deep. Stopping at 750K iterations, 393 V100-days in total, already gives FID $6.49$, under BigGAN-deep's $6.95$. At sampling time a GAN needs one network pass per image and ADM-G needs 25 to 250. A nearest-neighbour search in Inception-V3 feature space on a handful of guided samples found no copies of training images.

Limitations and what came next

The paper names two limitations. Sampling is still slower than a GAN's single pass; the authors point to Luhman and Luhman's distillation of DDIM into a one-step model as a direction, noting its samples were not yet competitive with GANs. And classifier guidance needs labels: the paper offers no way to trade diversity for fidelity on an unlabeled dataset. It suggests guiding an image generator with a noised CLIP model to condition on text captions.

Both directions were taken up within a year. Classifier-free guidance trains one diffusion model with and without the label and combines its two predictions, so no separate classifier is needed. GLIDE and then unCLIP (DALL-E 2) guided text-to-image diffusion models, and latent diffusion moved the process into an autoencoder's latent space, the basis of Stable Diffusion. DiT replaced the U-Net with a Transformer on the same ImageNet benchmarks, conditioning it with adaptive layer norm, the same scale-and-shift that AdaGN applies after group norm.