Flow Matching for Generative Modeling

Teach the model one arrow at a time.

A generative model is a machine that turns noise into data. Flow Matching trains one by regressing the velocity of a fixed noise-to-data path, with no diffusion process to reason about and no differential equation to solve while you train. Build a short tower of ideas and the whole method falls out, plus a straighter path that samples faster.

What if training a generative model meant nothing fancier than showing it which way to step, over and over?

Every modern image generator is, underneath, a way of solving one problem: you have a pile of samples from some distribution you cannot write down (faces, say, or photographs of dogs) and you want a machine that produces fresh ones. The trick that has dominated the last few years is to learn the journey from easy to hard. Start from pure Gaussian noise, which you can sample trivially, and learn a transformation that carries that noise onto the data. Diffusion models do this by setting up a stochastic process that slowly destroys data into noise, then learning to run it backward. It works beautifully, and it is also a lot of machinery: a noising process, a score to estimate, a schedule to tune, and a reverse-time stochastic differential equation to reason about.

Flow Matching, from Meta AI, throws most of that machinery out. The pitch fits in a sentence. Pick a fixed path that carries noise to data. Write down the velocity that travels along it. Train a network to copy that velocity by plain least-squares regression. There is no diffusion process, no score, no stochastic reversal, and no differential equation running inside the training loop. At generation time you take the learned velocity and follow it from noise to data with any off-the-shelf ODE solver.

The catch, and the reason the paper is clever rather than obvious, is that the velocity you actually want to copy is an average over the entire dataset and is hopeless to compute. The bulk of this post is the argument for why you can ignore that and regress a one-example stand-in instead, and get the same answer. We build it in order: what a flow is, the objective you wish you could write, why it is intractable, and the trick that makes it tractable. Then the payoff, a path borrowed from optimal transport that runs in straight lines and samples faster than diffusion.

A flow is a velocity field you follow

Picture every point in your space (every possible image, flattened to a vector in $\mathbb{R}^d$) and imagine attaching a little arrow to each one. The arrows can change over time. That time-varying field of arrows is a vector field, written $v_t(x)$: at time $t$ and location $x$ it tells you which way, and how fast, to move. Drop a particle into the field and let it ride: its position obeys an ordinary differential equation,

where $\phi_t(x)$ is where the particle that started at $x$ has drifted to by time $t$. The map $\phi_t$ is the flow. Run it on a whole cloud of starting points and the cloud gets stretched and reshaped. If we start the cloud as pure noise $p_0 = \mathcal{N}(0, I)$ and the flow reshapes it into the data distribution $p_1$ by time $t = 1$, we have a generative model. Model the field $v_t$ with a neural network and this is a Continuous Normalizing Flow (CNF).

One convention to fix now, because it trips people coming from diffusion. Here $t = 0$ is noise and $t = 1$ is data. Generation always runs forward in time, from $0$ to $1$. Diffusion papers usually go the other way (data at $t=0$, noise at $t=1$, then reverse), because they have a separate noising process to undo. Flow Matching has no such process to reverse, so it just points the clock the natural way.

Below is the picture to keep in your head for the rest of the post. A field of arrows, and noise particles riding it from a blob in the middle out onto the data. The whole game is learning that field.

CNFs are old and elegant, and almost nobody used them at scale. The reason is training. The classic way to fit a CNF is maximum likelihood, which needs you to solve the ODE (1) numerically on every training step to know where your samples went and how the density changed. Solving an ODE means many sequential evaluations of the network, and doing that inside every gradient step is painfully slow. People wanted a way to train the field directly, without simulating it. That is the door Flow Matching opens.

The objective you wish you could use

Suppose a kind oracle handed you the right velocity field: the field $u_t(x)$ that, when you follow it, carries noise exactly onto the data. Then training is the easiest thing in the world. Just regress your network $v_t$ onto it with squared error:

Read it straight: pick a random time $t$, pick a point $x$ from the cloud as it looks at that time (that is the $p_t(x)$ under the expectation), and push the network's arrow toward the oracle's arrow. Drive this loss to zero and your $v_t$ equals $u_t$ everywhere the cloud visits, so your flow generates the data. No ODE solve, no likelihood, no diffusion. This is the Flow Matching objective, and it is the cleanest training loss in generative modeling.

There is exactly one problem, and it is fatal as stated. We do not have the oracle. We have no idea what $u_t(x)$ is. There are infinitely many fields whose flow lands on the data, and even if we fixed a particular target path $p_t$ we want the cloud to follow, the field that generates it is an integral over the whole unknown data distribution. We cannot sample from $p_t(x)$ and we cannot evaluate $u_t(x)$. The dream objective (2) is uncomputable. The rest of the method is one long, satisfying maneuver to make it computable without changing its answer.

One example at a time

The escape begins by shrinking the problem. Forget the whole dataset for a moment and look at a single data point $x_1$. For that one point, define a simple path: a cloud that starts as the standard Gaussian at $t = 0$ and contracts onto a tight little blob centered at $x_1$ by $t = 1$. Because it is conditioned on one example, call it a conditional probability path, $p_t(x \mid x_1)$. The paper takes it to be Gaussian at every time:

with a mean $\mu_t(x_1)$ and a scalar standard deviation $\sigma_t(x_1)$ that we get to design. The only requirements are the two endpoints. At $t = 0$ every conditional path must be the same standard noise, so $\mu_0(x_1) = 0$ and $\sigma_0(x_1) = 1$. At $t = 1$ it must concentrate on its data point, so $\mu_1(x_1) = x_1$ and $\sigma_1(x_1) = \sigma_{\min}$, a width small enough that the blob is essentially the point $x_1$. Everything in between is free.

This is a thing we can actually touch. Sampling from $p_t(x \mid x_1)$ is one line, $x = \mu_t(x_1) + \sigma_t(x_1)\,\varepsilon$ with $\varepsilon \sim \mathcal{N}(0, I)$, and the velocity that generates it (we derive it shortly) is a tidy closed form. Below is one such path. Slide the time and watch the noise cloud march in and shrink onto a single $x_1$. Toggle the schedule to compare the optimal-transport path with the diffusion one, which take the same endpoints by different routes.

One conditional path is not a generative model. It only knows about one data point. But a dataset is a pile of data points, and we are about to add the piles up.

Average the easy fields into the hard one

Here is the bridge from one example back to the whole distribution. The conditional paths all start at the same noise and each ends at its own data point. If we mix them, weighting each by how likely its data point is under the data distribution $q(x_1)$, we recover a path over the whole dataset. That mixture is the marginal probability path:

Check the endpoints. At $t = 0$ every conditional path is the same standard Gaussian, so the mixture is too: $p_0 = \mathcal{N}(0, I)$. At $t = 1$ each conditional path is a spike at its $x_1$, so the mixture is a spike at every data point in proportion to how common it is, which is the data distribution itself, $p_1 \approx q$. So this mixture path is exactly the noise-to-data journey we wanted, assembled out of per-example pieces.

Now the part that is not obvious. Each conditional path has its own generating velocity $u_t(x \mid x_1)$. What single velocity field generates the mixture path? It would be lovely if it were just the average of the conditional velocities, and it almost is. You have to weight each conditional velocity by the posterior probability that the point $x$ came from data point $x_1$, which by Bayes is $p_t(x \mid x_1)q(x_1)/p_t(x)$:

This is the paper's first key result, and it is a clean one: the marginal velocity (5) generates the marginal path (4). The proof is two lines of the continuity equation, the conservation law that says a velocity $v_t$ generates a path $p_t$ exactly when $\partial_t p_t + \nabla\!\cdot(p_t v_t) = 0$. Differentiate the mixture (4) in time, swap the conditional velocity in using the fact that each $u_t(\cdot \mid x_1)$ generates its own $p_t(\cdot \mid x_1)$, pull the divergence outside the integral, and out pops the continuity equation for $u_t$ and $p_t$. The weighted average of the simple fields is the field that drives the mixture.

So the intractable oracle $u_t$ from the dream objective is not so mysterious after all. It is a posterior-weighted average of conditional velocities we can each write down. Drag the probe point below: each amber arrow is one data point's conditional push (fatter when that point is the more likely explanation of where the probe is), and the teal arrow is the average they produce, which is $u_t$ at that spot.

We are closer, but not done. The formula (5) still has the data distribution $q(x_1)$ and the marginal density $p_t(x)$ inside it, both unknown, both integrals over the whole dataset. We still cannot compute $u_t(x)$ to regress against. The last step is the one that feels like a magic trick.

The trick: regress the easy target, get the hard one for free

Instead of regressing against the intractable marginal velocity, regress against the per-example conditional velocity. Replace the oracle target $u_t(x)$ in the dream objective (2) with $u_t(x \mid x_1)$, and sample $x$ from that one example's conditional path. This is the Conditional Flow Matching (CFM) objective:

Every piece of this is computable. Draw a real data point $x_1$ from your dataset, draw a time $t$, draw a sample $x$ on that example's conditional path, and regress the network onto the closed-form conditional velocity. No marginal, no $q(x_1)$ integral, no $p_t(x)$. Just one example, one time, one arrow.

And the claim that makes it all work: the CFM objective (6) and the Flow Matching objective (2) have identical gradients in $\theta$. Minimizing the easy one is exactly minimizing the hard one. Your network converges to the marginal velocity $u_t$ even though you only ever showed it conditional velocities.

Why is that true? It is not a coincidence, it is a property of squared error. Expand both losses with $\|v - u\|^2 = \|v\|^2 - 2\langle v, u\rangle + \|u\|^2$. The $\|u\|^2$ term has no $\theta$ in it, so it drops out of the gradient. The $\|v\|^2$ term is identical in both losses once you note that averaging over the marginal $p_t(x)$ is the same as averaging over $q(x_1)$ and then $p_t(x \mid x_1)$, since one is the mixture of the other. The only term that could differ is the cross term $\langle v, u\rangle$, and plugging the definition of the marginal velocity (5) into it shows the marginal cross term and the conditional cross term integrate to the same thing. The two losses differ by a constant that does not depend on $\theta$, so

There is a one-sentence way to feel it without the algebra. Least-squares regression always converges to the conditional mean of its target. The target in CFM is the conditional velocity $u_t(x \mid x_1)$ for a random $x_1$, so the network converges to $\mathbb{E}[u_t(x \mid x_1) \mid x]$, the average conditional velocity over all the data points that could have produced this $x$. That average is the marginal velocity (5). You are training against single arrows and landing on their average, because that is what regression does. This is the same move that lets denoising score matching train a score without ever knowing the true score, generalized from scores to velocities.

That is the entire conceptual payload. Everything from here is choosing a good conditional path so the arrows are easy to learn and cheap to follow. But first, make the loop concrete.

A worked step

Take the optimal-transport path we are about to define, with $\sigma_{\min} = 0$ for cleanliness. The conditional path is the straight interpolation $x_t = (1-t)\,x_0 + t\,x_1$ from a noise sample $x_0$ to the data point $x_1$, and its velocity is the constant $x_1 - x_0$. One training step on a batch of $n$ images, each a $d$-vector:

# one Conditional Flow Matching step, OT path (sigma_min = 0)
x1 = sample_data()                 # a real data point     [n, d]
x0 = randn_like(x1)                # a noise sample        [n, d]
t  = rand(n, 1)                    # times ~ U[0,1]        [n, 1]
xt = (1 - t) * x0 + t * x1         # point on the straight path
target = x1 - x0                   # the constant velocity (the answer)
loss = mse(v(xt, t), target)       # regress the arrow
loss.backward()                    # no ODE solve, no diffusion

Concretely, for CIFAR-10 at $32\times32\times3$ a batch might be $x_1$ of shape $[256, 3072]$, $x_0$ the same shape of fresh Gaussian noise, $t$ of shape $[256, 1]$, the interpolation point $x_t$ of shape $[256, 3072]$, and the regression target $x_1 - x_0$ also $[256, 3072]$. The network $v$ (a U-Net) takes the noised image and the time and predicts a $[256, 3072]$ velocity. The loss is one mean-squared error. No ODE was solved, no noise schedule was consulted, no score was estimated. It is the simplest training loop in the family.

To generate, you do the one thing CFM never did during training: actually solve the ODE, forward from noise to data.

# generation: solve the ODE forward, t = 0 -> 1
x = randn(n, d)                    # start from pure noise (p0)
for t in linspace(0, 1, steps):    # any off-the-shelf ODE solver
    x = x + (1 / steps) * v(x, t)  # Euler step along the learned field
return x                           # x ~ data distribution (p1)

A formula for the target arrow

The worked step quietly used a closed form for the conditional velocity. Here is where it comes from, and it covers every Gaussian path at once. Take the Gaussian conditional path (3) and build the obvious flow that produces it: start with a standard normal sample and stretch it by the standard deviation, then shift it by the mean,

When $x \sim \mathcal{N}(0, I)$, the output $\psi_t(x)$ is exactly $\mathcal{N}(\mu_t, \sigma_t^2 I)$, so this affine map pushes the noise onto the conditional path. Its velocity is forced: differentiate the flow definition (1) and the inverse of the affine map, and the unique velocity that generates this path is

where the primes are time derivatives. This one formula is the workhorse. Hand it any mean schedule $\mu_t$ and width schedule $\sigma_t$ with the right endpoints and it spits out the exact velocity to regress. Two choices recover the diffusion world.

The variance-exploding path keeps the mean at the data point and grows the noise: $\mu_t(x_1) = x_1$, $\sigma_t(x_1) = \sigma_{1-t}$ for an increasing schedule. Plug into (9) and the mean term vanishes, leaving $u_t(x \mid x_1) = -\frac{\sigma_{1-t}'}{\sigma_{1-t}}\,(x - x_1)$. The variance-preserving path, the one behind DDPM, scales the mean down by $\alpha_{1-t}$ and the variance up to compensate, with $\alpha_t = e^{-\frac12 T(t)}$ and $T(t) = \int_0^t \beta(s)\,ds$. Plug those in and (9) reproduces the velocity of the probability-flow ODE that score-based diffusion already uses. So diffusion paths are not a rival to Flow Matching. They are points inside its menu, reachable by a particular pair of schedules $(\mu_t, \sigma_t)$.

That reframing is worth a beat. The whole apparatus of forward and reverse stochastic processes was one way to arrive at a Gaussian path. Flow Matching skips the process and writes the path down. Even when you choose the diffusion path, training it with the CFM velocity loss (instead of score matching) turns out more stable in the paper's experiments. And once you see the schedules as free parameters, you can ask for a path no diffusion produces.

The straight-line path

What is the simplest possible pair of schedules? Move the mean and the width in straight lines:

At $t = 0$ this is the standard Gaussian ($\mu = 0,\ \sigma = 1$); at $t = 1$ it is the blob at $x_1$ ($\mu = x_1,\ \sigma = \sigma_{\min}$). Both endpoints satisfied, by the most boring interpolation there is. Feed (10) into the velocity formula (9) and the time-derivatives are constants, giving

and the flow that carries a noise sample $x_0$ along it is the straight line $\psi_t(x_0) = \big(1 - (1 - \sigma_{\min})t\big)x_0 + t\,x_1$. Substitute into the CFM loss and the regression target collapses to a single constant vector,

The target $x_1 - (1-\sigma_{\min})x_0$ does not depend on $t$ at all. It is the arrow that points from the noise sample straight at the data sample, and the particle travels that arrow at constant speed. (With $\sigma_{\min} = 0$ it is just $x_1 - x_0$, the form the released libraries use.)

This is not an arbitrary choice. That straight-line flow $\psi_t$ is the optimal-transport displacement map between the two Gaussians: of all the ways to morph the prior into the data blob, this is the one that moves mass the shortest total distance, in straight lines, at constant speed. Optimal transport is the mathematics of moving a pile of sand to a new shape with the least total carrying, and McCann showed that between two Gaussians the answer is exactly this linear interpolation. The diffusion path, by contrast, takes a curved detour: it barely moves the sample early, then rushes it toward the data near the end, and its trajectories can swing past the target and double back.

Watch both. Each line is one noise sample being carried to the same $x_1$. The optimal-transport flow draws straight, evenly-paced lines. Toggle to diffusion and the same endpoints get connected by curves that bow out and overshoot.

A caution the paper is careful to state, and so will we. The conditional flow is optimal transport, but the marginal flow (the average over all data points) is not, in general, an optimal-transport map. Straight conditional lines do not guarantee straight marginal trajectories. Still, the marginal field stays relatively simple, and that simplicity is what pays off at sampling time.

Why straight is cheap to sample

Generation means solving the ODE (1) from $t = 0$ to $t = 1$, and a solver works by taking discrete steps. The cost is counted in function evaluations (NFE), one network call per step. Fewer steps means cheaper sampling, and how few you can get away with depends entirely on how curved the trajectory is.

A straight, constant-speed path is the easy case. A solver that assumes straight lines between steps tracks it almost perfectly even with a handful of steps. A curved path makes the same coarse solver cut corners: it steps along a chord, misses the bend, and drifts off the true trajectory, so you have to add steps to keep up. That is the practical edge of the optimal-transport path. The paper measures it directly: matching the same numerical error, Flow Matching with the OT path needs roughly 60% of the function evaluations the diffusion path needs.

See it below. We integrate the marginal field with $N$ Euler steps and check what fraction of particles land on the data. The optimal-transport field lands them with very few steps. Toggle to the curved diffusion-like field and the same step budget leaves particles stranded between clusters until you crank $N$ up.

So what does it actually do

The headline is that a loss this plain is competitive with, and often better than, the carefully-engineered diffusion training it replaces, on the same architecture. The authors take one U-Net and train it three ways: standard score matching with a diffusion path, Flow Matching with that same diffusion path, and Flow Matching with the optimal-transport path. On CIFAR-10 and ImageNet at 32, 64, and 128 the OT version wins across the board.

On ImageNet 32 the OT model reaches FID 5.02 against 5.68 for score matching and 6.99 for DDPM, while needing 122 function evaluations to score matching's 178 and DDPM's 262. (FID, the Fréchet Inception Distance, scores sample quality; lower is better. NFE counts the network calls an adaptive solver needs.) On CIFAR-10 it posts FID 6.35 at 2.99 bits per dimension. On ImageNet 64 it is FID 14.45 at 138 NFE. At 128 it reaches FID 20.9, the first CNF trained at that resolution. Better samples, better likelihoods, and fewer steps to produce them, all from the same network and the plainest possible loss.

Three things travel out of this paper. The CFM trick (regress per-example targets, converge to their average) is the engine, and it works for any path, not just Gaussians. The reframing of diffusion as one Gaussian path among many removed the mystique from a lot of generative modeling and let people design paths on purpose. And the straight optimal-transport path, with its constant-velocity target $x_1 - x_0$, became the default recipe almost everywhere downstream. When you read that a model was trained by "regressing $x_1 - x_0$ on the interpolation between noise and data," that is this paper, with $\sigma_{\min}$ set to zero. Stable Diffusion 3 and the current generation of large flow models are built on it.

The limits are honest. The conditional flow being optimal transport does not make the learned marginal flow optimal transport, so the trajectories are simpler but not perfectly straight, and very few sampling steps still cost you some quality. The Gaussian-path family, broad as it is, is still a family; richer paths (non-isotropic, non-Gaussian, on curved spaces) are left as future work the framework invites. None of that has slowed it down. Flow Matching took the most elegant generative model nobody could train and made it the easiest one to train, by noticing you never needed the hard target in the first place. You just needed to show the network one arrow at a time.