Flow Matching for Generative Modeling

Train a generative model by regressing the straight-line velocity from a noise sample to a data sample.

A continuous normalizing flow turns Gaussian noise into data by following a learned velocity field. Flow Matching trains that field with a least-squares loss on one training example at a time, and proves this gives the same gradients as matching the velocity of the whole dataset, which nobody can compute.

On ImageNet 32×32, one U-Net trained with the Flow Matching loss reaches FID 5.02 (an image-quality distance; lower is better) with 122 network calls per sample. The same U-Net trained with score matching reaches 5.68 with 178.

A generative model for images has to produce new samples from a distribution that nobody can write down. The common recipe starts from Gaussian noise, which is trivial to sample, and learns a map that carries noise onto the data. Diffusion models build that map by defining a process that gradually adds noise to images and training a network to undo it step by step. Their training loss targets the score, the gradient of the log-density of the noised data, and sampling runs a reverse-time stochastic or ordinary differential equation.

Flow Matching (Lipman, Chen, Ben-Hamu, Nickel and Le, 2022) trains the map by regression. You choose a path of distributions that goes from noise at time 0 to data at time 1, write down the velocity that moves samples along it, and fit a network to that velocity with squared error. Training never solves a differential equation. Sampling solves one, with any off-the-shelf ODE solver.

The velocity of the whole noise-to-data path depends on the entire dataset at once and cannot be computed. The paper proves that you can regress a per-example velocity instead, which is available in closed form, and get the same gradients. The page follows the paper's order of ideas: what a flow is, the objective you would like to minimize, the per-example path and its velocity, how the per-example velocities average into the dataset velocity, why regressing them works, and the straight optimal-transport path that makes sampling cheaper. Equation numbers on this page are the paper's.

A flow is a velocity field you follow

Flatten an image into a vector in $\mathbb{R}^d$ (for a 32×32 RGB image, $d$ is 3072). A time-dependent vector field $v_t(x)$ assigns a velocity to every point $x$ at every time $t$ in [0, 1]. A particle dropped into the field moves according to an ordinary differential equation:

The quantity $\phi_t(x)$ is the position at time $t$ of the particle that started at $x$, and the map $\phi_t$ is the flow. Apply it to a whole cloud of starting points drawn from $p_0$ and the cloud moves and changes shape; its distribution at time $t$ is written $p_t$. If $p_0$ is the standard Gaussian and $p_1$ is the data distribution, the flow is a generator: sample noise, integrate (1) from 0 to 1, and read off an image. When $v_t$ is a neural network, the model is a continuous normalizing flow (CNF), introduced with Neural ODEs.

The time convention is the reverse of most diffusion papers. Here $t = 0$ is noise and $t = 1$ is data, and generation integrates forward in time. Diffusion papers put data at 0 and noise at the end because they define a forward noising process first and then reverse it; Flow Matching defines no noising process, so it orders time in the direction of generation. The diffusion formulas below carry a $1 - t$ for this reason.

Press the button to watch 150 noise samples ride a field whose particles end on four data clusters. The faint arrows are the field at the current time; they change as $t$ advances.

CNFs existed before this paper, but they were trained by maximum likelihood. Computing the likelihood of a training image means integrating (1) numerically, together with the change in log-density along the way, and every step of the ODE solver is a forward pass of the network. A single gradient step therefore costs dozens to hundreds of sequential network evaluations, and backpropagating through the solver costs as much again. The paper calls its alternative simulation-free: the loss is computed from one network evaluation per training example, with no ODE solve.

The Flow Matching objective

Suppose someone gave you a target probability path $p_t$ from noise to data and the vector field $u_t$ that generates it (following $u_t$ with (1) turns samples of $p_0$ into samples of $p_t$ at every $t$). Training would be a regression:

Sample a time uniformly, sample a point $x$ from the path at that time, and penalize the squared distance between the network's velocity and the target velocity. At zero loss the network's field equals $u_t$ wherever $p_t$ has mass, so its flow reproduces the path and lands on the data. This is the Flow Matching (FM) objective.

As written, it cannot be computed. Neither $p_t$ nor $u_t$ is given. Many paths lead from noise to a given data distribution, and for any specific one, $u_t(x)$ depends on where every training image is relative to $x$, so there is no formula for it and no way to sample $p_t$ directly. The remaining sections build a path for which $p_t$ is easy to sample, show what its $u_t$ is, and then replace $u_t$ in (5) with a target that is computable and gives the same gradients.

A single data point: the conditional path

Start with a single training example $x_1$. For that one point, define a path of Gaussians that begins as the standard normal at $t = 0$ and ends as a narrow Gaussian centered on $x_1$ at $t = 1$. The paper calls it a conditional probability path, conditional because it depends on the chosen $x_1$:

The mean $\mu_t(x_1)$ and the scalar standard deviation $\sigma_t(x_1)$ are schedules you choose, subject to the endpoints. At $t = 0$ every conditional path must be the same noise distribution, so $\mu_0 = 0$ and $\sigma_0 = 1$. At $t = 1$ the path must sit on its data point, so $\mu_1 = x_1$ and $\sigma_1 = \sigma_{\min}$, a small positive width. Between the endpoints any smooth schedule is allowed; the diffusion paths and the optimal-transport path later on differ only in this choice.

The simplest flow that produces this path scales a standard-normal sample by $\sigma_t$ and shifts it by $\mu_t$:

If $x_0$ is standard normal, $\psi_t(x_0)$ has mean $\mu_t$ and standard deviation $\sigma_t$, which is (10). Its velocity follows from differentiating (11) in $t$ and expressing the result at the current position instead of the starting point. The time derivative of $\psi_t(x_0)$ is $\sigma_t' x_0 + \mu_t'$, and inverting (11) gives $x_0$ from the current position $x$ as $(x - \mu_t)/\sigma_t$. Substituting, Theorem 3 of the paper states that the unique vector field generating (10) through (11) is

where primes are time derivatives. The second term, $\mu_t'$, is the velocity of the mean and is the same for every point. The first term is proportional to the point's offset from the mean; while $\sigma_t$ shrinks, $\sigma_t'$ is negative and the term pulls each point toward the mean at a rate of $|\sigma_t'|/\sigma_t$ per unit of offset.

A worked example with numbers, using the straight-line schedule the paper recommends (defined in full below) with $\sigma_{\min}$ set to 0: $\mu_t$ is $t x_1$ and $\sigma_t$ is $1 - t$. Take a data point $x_1$ = (2, 1), a noise sample $x_0$ = (−0.6, 0.8), and $t$ = 0.5. Then $\mu_t$ = (1, 0.5), $\sigma_t$ = 0.5, and the noisy point is:

With $\sigma_t'$ = −1 and $\mu_t'$ = $x_1$, formula (15) gives

which equals $x_1 - x_0$, the displacement from the noise sample to the data point. On this schedule every point of the conditional path moves toward its data point along a straight line at that constant velocity.

In Figure 2 the teal cloud is 420 samples of (10), all drawn with the same frozen noise so that sliding $t$ moves each sample continuously. The short arrows are (15) evaluated at a subset of the samples; the strip at the bottom plots the two schedules. Compare the arrows under the two schedules at $t$ = 0.5.

Under the OT schedule the arrows at $t$ = 0.5 point from each sample toward $x_1$ with similar lengths. Under the diffusion schedule at the same $t$, the mean has covered 28% of the distance to $x_1$ and the standard deviation is still 0.96, so the arrows are short. Most of the motion happens after $t$ = 0.9, where $\sigma_t'/\sigma_t$ grows large and the arrows hit the length cap.

The marginal field is a weighted average

A single conditional path generates copies of one image. To cover the dataset, mix the conditional paths, weighting each by the data distribution $q(x_1)$ (for a training set, each image with equal weight). The mixture is the marginal probability path:

At $t = 0$ every component is the standard normal, so $p_0$ is the standard normal. At $t = 1$ each component is a narrow Gaussian of width $\sigma_{\min}$ on its own data point, so $p_1$ is the data distribution blurred by $\sigma_{\min}$, which the paper writes as $p_1 \approx q$. Sampling $p_t$ is easy: pick a training image, pick noise, apply (11).

The velocity field that generates the mixture is a weighted average of the conditional fields (15). The weight on data point $x_1$ at position $x$ is the posterior probability that a sample found at $x$ came from the path aimed at $x_1$, which by Bayes' rule is $p_t(x \mid x_1)\,q(x_1)/p_t(x)$:

Theorem 1 of the paper states that the marginal vector field (8) generates the marginal path (6). The proof uses the continuity equation, the conservation law that a velocity field $u_t$ generates a path $p_t$ exactly when

The operator $\nabla\cdot$ is the divergence, the net outflow of a vector field from a point, and $p_t u_t$ is the flux of probability mass; the equation says density at a point falls exactly as fast as mass flows out of it. Differentiate the mixture (6) in time. Each conditional pair satisfies its own continuity equation, so the time derivative of each component equals minus the divergence of the component times its own conditional velocity. Pulling the divergence out of the integral over $x_1$ leaves the divergence of the integral in the numerator of (8), and by (8) that integral equals $p_t(x)\,u_t(x)$.

Let the data be two equally likely points, −2 and 2, use the OT schedule with $\sigma_{\min}$ = 0, and evaluate at $t$ = 0.5 and $x$ = 0.4. The conditional paths are Gaussians with means −1 and 1 and standard deviation 0.5. The point 0.4 is 1.2 standard deviations from the mean 1 and 2.8 from the mean −1, so the posterior weights are proportional to $e^{-0.72}$ and $e^{-3.92}$, which normalize to 0.961 and 0.039. The conditional velocities are (2 − 0.4)/0.5 = 3.2 toward the data point at 2 and (−2 − 0.4)/0.5 = −4.8 toward the one at −2. The marginal velocity is 0.961 × 3.2 + 0.039 × (−4.8) = 2.89: mostly the push toward 2, reduced slightly by the unlikely alternative.

Figure 3 does the same computation in 2D with four data points. Drag the probe and watch the weights, printed next to each data point, shift as the probe moves; slide $t$ toward 0 and the weights even out, toward 1 and one of them goes to 1.

Formula (8) still cannot be evaluated for a real dataset. The weights need $p_t(x)$, an integral over every training image, and for millions of images in 3072 dimensions the sum is too expensive to compute on every training step.

How Flow Matching trains: regress one example's velocity

Conditional Flow Matching replaces the target in (5). Instead of the marginal velocity at $x$, it regresses the conditional velocity of the data point that produced $x$, and it samples $x$ from that data point's conditional path. The paper names this the Conditional Flow Matching (CFM) objective:

Every term can be sampled or computed. Draw a training image $x_1$, a time $t$, and a noise sample $x_0$; set $x$ to $\psi_t(x_0)$; the target is (15), a closed-form function of $x$, $x_1$ and $t$. The same noisy $x$ can be produced by different data points, so the target for a given $x$ is random: in the two-point example above, a training pair at $x$ = 0.4 carries the target 3.2 about 96% of the time and −4.8 about 4% of the time.

Theorem 2 of the paper: assuming $p_t(x) > 0$ for all $x$ and all $t$ in [0, 1], the losses (9) and (5) are equal up to a constant that does not depend on $\theta$, so

The proof expands both squared errors,

and compares the three terms. The $\|u\|^2$ terms contain no $\theta$ and drop out of the gradient. The $\|v\|^2$ terms are equal because sampling $x_1$ from $q$ and then $x$ from $p_t(x \mid x_1)$ produces exactly the mixture $p_t(x)$. For the cross term, the FM version contains $u_t(x)$; substitute its definition (8), and the $p_t(x)$ in the denominator cancels the $p_t(x)$ the expectation integrates against, leaving the CFM cross term. The positivity assumption keeps that division defined.

A shorter route to the same conclusion uses a property of least squares: the function that minimizes expected squared error to a random target is the conditional mean of that target. For a fixed $x$ and $t$, the CFM targets are the conditional velocities of all the data points that could have produced $x$, drawn with the posterior probabilities of (8). Their mean is (8). So a network trained on single-example targets converges to the dataset-level field without ever computing it. The paper describes CFM as inspired by denoising score matching, which uses the same argument to learn a score from per-example targets.

Figure 4 checks the theorem numerically in 1D, with three data points at −2, 0.5 and 2. The top panel shows the training pairs a CFM batch would contain at the chosen $t$; each data point's pairs fall on its own line, and the marginal field runs between the lines. The bottom panel plots both losses for a model that equals the marginal field plus a constant offset $\delta$. Move $\delta$ and the two loss values change together; move $t$ and only the gap between them changes.

The gap $C_t$ is the variance of the CFM target around its conditional mean, averaged over $x$. At $t$ = 0 every data point is equally likely to be the source of any $x$, and $C_0$ equals the variance of the three data values, 2.72. It peaks near $t$ = 0.2 at 3.63 and falls to 0.04 by $t$ = 0.8, where the three conditional paths barely overlap. In training, this means the CFM loss never reaches zero: its floor is the average of $C_t$ over $t$, and only the part above that floor measures the model's error.

A training step and a sampling loop

With the straight-line schedule and $\sigma_{\min}$ = 0, the whole method fits in a few lines. The shapes are those of CIFAR-10 at the paper's batch size of 256. The network is a U-Net (the architecture from the Diffusion Models Beat GANs paper, with minimal changes) that takes the noisy images and the times and returns a tensor of the same shape as the images.

# one CFM training step on the OT path, CIFAR-10 shapes
# (paper Table 3: batch 256, Adam, lr 5e-4). sigma_min = 0 as in the library.
x1 = next(loader)                  # real images        [256, 3, 32, 32]
x0 = torch.randn_like(x1)          # noise              [256, 3, 32, 32]
t  = torch.rand(256)               # t ~ U[0, 1]        [256]
tb = t.view(-1, 1, 1, 1)           # for broadcasting   [256, 1, 1, 1]
xt = (1 - tb) * x0 + tb * x1       # psi_t(x0), eq. 22  [256, 3, 32, 32]
target = x1 - x0                   # eq. 23; no t in it [256, 3, 32, 32]
loss = ((unet(xt, t) - target) ** 2).mean()
loss.backward()                    # gradients reach only the U-Net weights
opt.step(); opt.zero_grad()

One step draws 256 images, 256 noise tensors and 256 times, forms 256 points on 256 straight lines, and regresses the U-Net output onto 256 displacement tensors of shape [3, 32, 32]. The gradient flows into the U-Net weights only; the target is a fixed tensor. The paper trains CIFAR-10 this way for 391k iterations with Adam at learning rate 5e-4. An ODE is solved only for generation and for likelihood evaluation. Generation integrates (1) with Euler steps:

# sampling: integrate dx/dt = v(x, t) from t = 0 to t = 1 (eq. 1)
x = torch.randn(64, 3, 32, 32)     # x0 ~ N(0, I), 64 samples
N = 20                             # Euler steps = network calls
for i in range(N):
    t = torch.full((64,), i / N)
    x = x + (1 / N) * unet(x, t)   # one Euler step along the field
# x now holds 64 generated images. Table 1 in the paper uses dopri5,
# an adaptive solver (tolerance 1e-5) that picks its own step count.

Diffusion paths as special cases

Formula (15) accepts any mean and width schedules with the right endpoints. Two choices reproduce the Gaussian paths of existing diffusion models, with $1 - t$ in place of $t$ because diffusion runs time the other way.

The variance-exploding (VE) path keeps the mean at the data point and grows the noise: $\mu_t$ is $x_1$ and $\sigma_t$ is $\sigma_{1-t}$ for an increasing schedule $\sigma$ with $\sigma_0$ = 0 and a large $\sigma_1$. The paper reuses the symbol $\sigma$ for this diffusion schedule, and because $\sigma_1$ is large rather than 1, the VE path starts at $t = 0$ as a wide Gaussian around $x_1$ instead of exactly $\mathcal{N}(0, I)$. The mean is constant, so only the first term of (15) survives:

The variance-preserving (VP) path, the one behind DDPM, shrinks the data toward zero while adding noise, so that data scaled to unit variance keeps unit variance along the path:

The paper uses the linear $\beta$ schedule from 0.1 to 20. At $t$ = 0.5 that gives $\alpha_{1-t}$ = 0.28 and $\sigma_t$ = 0.96; at $t$ = 0.9, $\alpha_{1-t}$ = 0.95 and $\sigma_t$ = 0.32. Plugging (18) into (15) gives a field (the paper's eq. 19) that coincides with the probability-flow ODE of score-based SDEs, the deterministic ODE whose solutions have the same marginal densities as the stochastic diffusion.

Diffusion models, seen this way, are one family of Gaussian paths inside the Flow Matching framework, reached by a particular pair of schedules. The stochastic forward process was one way to arrive at those schedules; Flow Matching writes the path down directly. The loss is a separate choice from the path. The paper trains the same U-Net on the VP path with both the FM velocity loss and the score-matching loss, and reports that the velocity loss trains more robustly and stably. Its FID is lower than score matching's on CIFAR-10 (8.06 against 19.94) and ImageNet 64 (16.88 against 19.74), and higher on ImageNet 32 (6.37 against 5.68).

The optimal-transport path

With the schedules free, the obvious candidate is to move both linearly in time:

At $t = 0$ this is the standard normal and at $t = 1$ a Gaussian of width $\sigma_{\min}$ on $x_1$. Both derivatives are constants, $\mu_t'$ is $x_1$ and $\sigma_t'$ is $-(1 - \sigma_{\min})$, and (15) simplifies to

The flow (11) that carries a noise sample along this path is

Evaluate (21) at $x = \psi_t(x_0)$: the numerator becomes $\sigma_t\,(x_1 - (1-\sigma_{\min})x_0)$, and the $\sigma_t$ cancels the denominator. Substituting into (9) gives the loss the paper trains with:

The target does not depend on $t$. A noise sample moves along a straight line to its data point at constant speed, and the regression target is that line's direction and length. With $\sigma_{\min}$ = 0 the flow is the interpolation $(1-t)\,x_0 + t\,x_1$ and the target is $x_1 - x_0$, the form the official library and the training code above use. That special case is the objective of Rectified Flow (Liu, Gong and Liu), and stochastic interpolants (Albergo and Vanden-Eijnden) reached a similar one; the paper cites both as concurrent work. The paper's general form keeps $\sigma_{\min}$ positive, so each conditional path ends in a Gaussian around $x_1$ rather than at a single point; the difference in the target is $\sigma_{\min} x_0$.

The path is named for optimal transport, the problem of moving one distribution onto another with the least total squared distance traveled. McCann (1997) showed that between a standard Gaussian and another Gaussian the optimal map moves each point in a straight line at constant speed, and (22) is that map. For the conditional paths, the OT schedule is therefore the cheapest way to turn noise into the Gaussian around $x_1$. The VP path joins the same endpoints along curved trajectories: in the $(x_0, x_1)$ plane the point $\alpha x_1 + \sqrt{1-\alpha^2}\,x_0$ travels a quarter ellipse, and when $x_0$ and $x_1$ point the same way it passes beyond $x_1$ and comes back.

Figure 5 draws fourteen conditional trajectories to one data point under each schedule, with a dot moving along each at the schedule's speed. Look for the dots' spacing under OT and the loops past $x_1$ under diffusion.

These are conditional trajectories. The network learns the marginal field (8), the posterior-weighted average of many straight conditional fields, and the paper states that this marginal flow is not, in general, an optimal-transport map. Its trajectories bend where the posterior weights change, as the particles in Figure 1 do early on while they choose a cluster. The paper's argument for the OT path is that its marginal field is simpler to fit and to integrate than the diffusion field, which the next section measures.

Why straight paths need fewer solver steps

Sampling integrates (1) with a numerical solver, and each solver step calls the network at least once. The count of calls is the number of function evaluations (NFE). The Euler method takes the velocity at the current point and moves in a straight line for one step, so its error per step grows with how much the trajectory curves and how fast the velocity changes over that step. A trajectory that is a straight line at constant speed is integrated exactly by one Euler step; a curved one needs steps short enough that each chord stays near the curve.

Figure 6 runs this on a toy where both marginal fields are known exactly: four tight Gaussian clusters, the OT path with $\sigma_{\min}$ = 0 and the VP path with the paper's schedule. It integrates 120 fixed noise samples with $N$ Euler steps and measures each endpoint's distance from an accurate solution. Start at $N$ = 4 and compare the two readouts, then drag to $N$ = 1.

With 4 steps the OT samples end 0.144 from the exact endpoints on average and the diffusion samples 0.601; diffusion needs 9 steps to get below 0.144. To match OT's error at 10 steps (0.059) it needs 16. By 30 steps both are within 0.03 and the gap is small. Switch to the diffusion field at $N$ = 40 to see the marginal trajectories: several bend sharply late in the flow, and a few swing past their cluster before settling.

At $N$ = 1 the OT field fails. A single Euler step evaluates the field only at $t$ = 0, where every conditional path is the same standard normal, so every data point has the same posterior weight and (8) points each sample at the average of the data. The VP field at $t$ = 0 is close to zero because its schedule barely moves early, so one step leaves the samples as noise.

The paper measures the same effect on real models. With fixed-step solvers on the ImageNet 32×32 models, comparing per-pixel error against 1000-step solutions over 256 noise seeds, the FM-OT model needs roughly 60% of the function evaluations the diffusion-path models need to reach the same error (Figure 7 of the paper, midpoint solver, budgets of 100 NFE or fewer). With the adaptive solver used for Table 1, the OT model also takes the fewest steps: 122 against 178 for score matching and 262 for DDPM on ImageNet 32.

Results

The main comparison trains one U-Net with five objectives on CIFAR-10 and ImageNet at 32×32 and 64×64: DDPM, score matching and ScoreFlow on the VP diffusion path, Flow Matching on the diffusion path, and Flow Matching on the OT path. FM-OT, FM-Diffusion and score matching share the architecture, hyperparameters and number of epochs; the paper notes that the other baselines were allowed more iterations. Three numbers are reported per model: negative log-likelihood in bits per dimension (the cost in bits of encoding each pixel value under the model; lower is better), FID (Fréchet Inception Distance, a distance between the feature statistics of generated and real images; lower is better), and NFE for the adaptive solver dopri5 at tolerance 1e-5, averaged over 50k samples.

FM-OT is best on all three measures in all three settings. The likelihood gains are small (0.01 bits per dimension over the next model on ImageNet 32, 0.10 on CIFAR-10); the FID and NFE gains are larger. On ImageNet 64 the FID drops from 16.88 with the diffusion path to 14.45 with the OT path under the same loss, and the solver needs 138 evaluations instead of 187, so the path choice alone accounts for about a quarter fewer network calls per sample. The paper attributes the high CIFAR-10 FIDs of all models to an architecture not tuned for CIFAR-10.

At 128×128 the paper trains only FM-OT, so there is no same-architecture baseline. It reaches FID 20.9 and NLL 2.90, and the paper compares against unconditional GANs, calling the FID state of the art except for IC-GAN, which conditions on features from a self-supervised ResNet-50. Training was also cheaper: Dhariwal and Nichol's diffusion model for ImageNet 128 trains for 4.36M iterations at batch 256, while FM, with a 25% larger model, used 500k iterations at batch 1.5k, about 33% fewer images seen. The paper also reports that FM-OT lowers FID faster during training on ImageNet 64 and that its sampling cost stays constant as training proceeds, while the cost of sampling a score-matching model changes a lot over training.

A last experiment uses FM-OT for 4× super-resolution, 64×64 to 256×256, conditioned on the low-resolution image. Against SR3 it reaches FID 3.4 against 5.2 and Inception Score 200.8 against 180.1, with slightly lower PSNR (24.7 against 26.4) and SSIM (0.747 against 0.762). A plain regression baseline has the best PSNR, 27.9, and the worst FID, 15.2; the paper follows SR3 in treating FID and IS as the better measure of generation quality.

Limits

The OT property holds for each conditional flow, not for the learned marginal flow, so the marginal trajectories curve and low step counts still cost accuracy (Figure 6 at small $N$). The experiments cover Gaussian conditional paths with isotropic noise; the conclusion names non-isotropic Gaussians and more general kernels as directions the framework allows. The comparisons use one architecture on unconditional ImageNet up to 64×64, with the 128×128 result compared only against GANs.

The $\sigma_{\min}$ = 0 version of (23), regressing $x_1 - x_0$ on the straight interpolation between a noise sample and a data sample, is the objective Stable Diffusion 3 (Esser et al., 2024) trains a transformer with, under the name rectified flow, in the compressed latent space introduced by Latent Diffusion. SD3 also draws the training times from a non-uniform distribution. That system is outside the paper; the paper's own models are pixel-space U-Nets.