TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate

A random rotation lets one fixed quantizer compress any vector to within 2.7× of the best possible error.

TurboQuant rotates each vector by a random orthogonal matrix, rounds every coordinate to the nearest entry of a small precomputed table, and adds a 1-bit correction when inner products have to come out unbiased. It needs no training data and no calibration.

The paper also proves a lower bound that holds for any quantizer, keeps a Llama-3.1-8B KV cache at 3.5 bits per channel with no drop in LongBench average, and encodes 100,000 search vectors in 0.7 to 2.1 milliseconds.

A language model's KV cache stores one key vector and one value vector per token, per attention head, per layer. For Llama-3.1-8B a head vector has 128 numbers; in 16-bit floating point that is 2,048 bits, and a 100,000-token context holds millions of them. A vector database for retrieval-augmented generation holds millions of embeddings of 1,536 or 3,072 numbers each. Both are expensive in memory, and on a GPU the time to move them from memory to the arithmetic units often costs more than the arithmetic. Storing each number in 2 to 4 bits instead of 16 cuts both costs by 4 to 8 times, if the vectors survive the rounding.

TurboQuant (Zandieh, Daliri, Hadian and Mirrokni, from Google Research, Google DeepMind and NYU) is a vector quantizer with three properties at once. It is online: the same fixed map applies to each vector the moment it arrives, with nothing fitted to the data. Its error is provably within a constant factor of the least error any quantizer can reach at the same number of bits. And it runs as a matrix multiply plus a table lookup, which a GPU does quickly. The page builds it in the order the paper does: what error to measure, why a random rotation makes one fixed quantizer sufficient, how to quantize a single coordinate, how close that gets to the information-theoretic floor, and why inner products need a second, 1-bit stage.

What a vector quantizer has to preserve

The input is a vector $\mathbf{x}$ with $d$ coordinates and a budget of $b$ bits per coordinate, the bit-width. A quantizer is a pair of maps: an encoder $Q$ that turns $\mathbf{x}$ into a code of $b \cdot d$ bits, and a decoder $Q^{-1}$ that turns the code back into a reconstruction $\tilde{\mathbf{x}}$. TurboQuant's encoder is randomized (it draws a random rotation once), so its errors are stated as expectations over that randomness, for a fixed input.

The first error measure is the squared distance between the vector and its reconstruction, the mean-squared error:

Most uses of these vectors are dot products. Attention scores a query against every cached key by their inner product (see Attention Is All You Need), and a nearest-neighbour search ranks database vectors by their inner product with a query. So the second measure is the squared error of an inner product with an arbitrary query $\mathbf{y}$:

For inner products the paper also asks that the estimate be unbiased: averaged over the quantizer's randomness, it equals the true value exactly.

Squared error and bias behave differently when estimates are combined. If a thousand inner products each carry independent zero-mean noise, the noise in their sum grows as $\sqrt{1000}$ while the sum grows as 1000, so the relative error of the sum is about 32 times smaller than that of one term. If each carries the same bias, say every estimate is 10% too small, the sum is 10% too small no matter how many terms there are. The paper therefore gives two algorithms: one minimizes (1), and the other keeps (2) small while staying unbiased.

Two constraints shape the design. The input is worst-case: the guarantees must hold for every unit vector, with no assumption that the data follows a friendly distribution. And the quantizer is data-oblivious, which the paper also calls online: it cannot look at the dataset first. The usual alternative, product quantization, fits a codebook to the data with k-means, which needs a preprocessing pass over the whole dataset and has to be redone when the data changes. A KV cache changes with every generated token, so a quantizer for it has to encode each new key and value immediately, with the same fixed map.

The theorems assume $\lVert\mathbf{x}\rVert = 1$. A vector of any other length is stored as its floating-point norm plus the quantized unit vector, and rescaled after decoding. For a 128-dimensional key at $b = 3$, the code is $3 \times 128 = 384$ bits plus one norm, against 2,048 bits in 16-bit floating point, about 5 times smaller.

Why TurboQuant rotates every vector first

The simplest vector quantizer rounds each coordinate on its own against a fixed list of allowed values. That list has to be chosen for some expected spread of coordinate values, and a worst-case input can always violate it. Take $d = 64$ and a unit vector whose length is all in one coordinate: one entry equals 1 and the other 63 are 0. A list designed for coordinates of typical size $1/\sqrt{64} = 0.125$ has its largest entry near 0.19 (the exact 2-bit values come in the next section), so the 1 is stored as 0.19, and the zeros are stored as $\pm 0.057$ because the list has no zero. The squared error is 0.66 from the spike plus 0.20 from the 63 zeros, 0.86 in total. At one bit it is 1.44, worse than storing nothing (reconstructing the zero vector costs exactly 1). A list designed for spikes would do badly on a vector spread evenly across all coordinates.

TurboQuant multiplies $\mathbf{x}$ by a random rotation $\boldsymbol{\Pi}$ before quantizing, and by $\boldsymbol{\Pi}^{\!\top}$ after decoding. A random rotation is an orthogonal matrix drawn uniformly; the paper builds one from the QR decomposition of a $d \times d$ matrix of i.i.d. Gaussian entries. Orthogonal matrices preserve every length and every angle, so $\langle \boldsymbol{\Pi}\mathbf{y}, \boldsymbol{\Pi}\mathbf{x}\rangle = \langle \mathbf{y}, \mathbf{x}\rangle$ and the squared error is the same measured before or after the rotation. What changes is the coordinates. For any fixed unit vector $\mathbf{x}$, the rotated vector $\boldsymbol{\Pi}\mathbf{x}$ is a uniformly random point on the unit sphere. The spike above becomes a vector whose 64 coordinates all have typical size 0.125, and so does every other input. The same $\boldsymbol{\Pi}$ is used for every vector, so it is drawn once and costs nothing per vector.

Lemma 1 of the paper gives the exact distribution of one coordinate of a uniform point on the unit sphere in $d$ dimensions:

The shape follows from geometry. The points of the sphere whose first coordinate equals $x$ form a smaller sphere of radius $\sqrt{1-x^2}$ in one fewer dimension. The density is proportional to that sphere's surface area, $(1-x^2)^{(d-2)/2}$, times $1/\sqrt{1-x^2}$ for the slope of the sphere at that height, which gives $(1-x^2)^{(d-3)/2}$, the shape of a rescaled Beta distribution. Its mean is 0 because the density is symmetric about zero. Its variance is $1/d$, because the $d$ squared coordinates of a unit vector sum to 1 and all have the same distribution. As $d$ grows the density approaches a Gaussian $\mathcal{N}(0, 1/d)$; at $d = 128$ the standard deviation is 0.088.

In Figure 1, the top row shows an input vector and one random rotation of it, and the histogram pools the rotated coordinates over many rotations. Switch between the three input shapes and the top-left stems change completely while the histogram stays on the same curve. Lower $d$ to 4 to see the Beta shape, which at small $d$ is visibly flatter than a Gaussian.

The per-coordinate approach also needs the coordinates to be close to independent of each other, so that quantizing them one at a time loses little against quantizing them jointly. They cannot be exactly independent, since their squares sum to 1. In high dimensions they are nearly independent: Diaconis and Freedman (1987) showed that any $k$ coordinates of a uniform point on the sphere in $d$ dimensions are close to $k$ independent Gaussians, with an error that grows like $k/d$. The statement is an approximation that holds while $k$ is much smaller than $d$, and the paper's analysis uses it in that high-dimensional sense.

How TurboQuant quantizes one coordinate

After the rotation, every coordinate of every input is a draw from the same density $f_X$. The design problem reduces to one dimension: choose $2^b$ values, the centroids $c_1 \le \cdots \le c_{2^b}$, and round each coordinate to the nearest one, so that the expected squared error is as small as possible. With a known density, this is the problem the Lloyd-Max quantizer solves, k-means on a continuous distribution in one dimension. An optimal solution satisfies two conditions. Each value goes to its nearest centroid, so cell boundaries sit at the midpoints between neighbouring centroids. And each centroid is the mean of the values that fall in its cell, since the mean minimizes squared error within a cell. The paper writes the problem as

and solves it numerically, once, for each bit-width it needs. Lloyd's algorithm alternates the two conditions (move boundaries to midpoints, move centroids to cell means) until nothing changes, and produces a table of $2^b$ numbers that every vector shares. For large $d$, in units of $1/\sqrt{d}$, the one-bit centroids are $\pm\sqrt{2/\pi} = \pm 0.798$ and the two-bit centroids are $\pm 0.453$ and $\pm 1.51$.

A worked example at $d = 128$, $b = 2$: the centroids are $\pm 0.453/\sqrt{128} = \pm 0.040$ and $\pm 1.51/\sqrt{128} = \pm 0.134$, with cell boundaries at 0 and $\pm 0.087$. A rotated coordinate equal to 0.100 lies above 0.087, so it gets the index of the top centroid, is stored as the 2-bit code 11, and decodes to 0.134, an error of 0.033. A coordinate equal to −0.020 decodes to −0.040.

In Figure 2, the curve is $f_X$ in units of $1/\sqrt{d}$, the amber ticks are the centroids and the dashed lines are the cell boundaries. Click or drag on the plot to place a value and read its error. At one bit the quantizer keeps only the sign of the coordinate; each added bit doubles the number of cells.

Why Lloyd-Max levels and not evenly spaced ones, or levels at equal-probability quantiles? Evenly spaced levels waste resolution in the tails, where few coordinates fall. Quantile levels, which is how QLoRA's NF4 format spaces its 16 values for normally distributed weights, give every level the same share of the values, which maximizes the entropy of the index but does not minimize squared error. Lloyd-Max levels minimize squared error, which is the quantity both theorems bound. Their indices are then not equally likely: at $b = 4$ the index distribution has entropy about 3.8 bits, so an entropy coder could shrink the code by about 5%. The paper leaves that out to keep encoding simple and fast.

Put together, the MSE quantizer is Algorithm 1 of the paper:

# TurboQuant_mse (Algorithm 1): b bits per coordinate, d-dim unit vector x
# set-up, once per (d, b), shared by every vector:
Pi = qr(randn(d, d)).Q              # random rotation
cb = lloyd_max(b) / sqrt(d)         # 2^b sorted centroids, solved offline

def quant(x):                       # x is unit norm; store ||x|| separately
    y = Pi @ x                      # each y[j] ~ N(0, 1/d) for large d
    return argmin_k |y[j] - cb[k]|  # one b-bit index per coordinate

def dequant(idx):
    return Pi.T @ cb[idx]           # look up centroids, rotate back

Figure 3 runs exactly this on a 64-dimensional unit vector, with the rotation switched on or off. The top panel shows the numbers the quantizer rounds, with the $2^b$ allowed values as dashed lines. The bottom panel shows the decoded vector in the original coordinates. With the one-spike input and the rotation off, the spike sits far above every allowed value. Switch the rotation on and the error drops from 0.86 to about the average 0.117 at 2 bits, and draw new rotations to see how little it varies. The ramp input shows the other side: its unrotated coordinates happen to fit the table well (0.077 at 2 bits), and the rotation gives that up for the same 0.117 on every input.

How close TurboQuant gets to the optimal distortion

All $d$ rotated coordinates have the same distribution, so the vector's expected squared error is $d$ times the error of one coordinate, $D_{\tt mse} = d\cdot\mathcal{C}(f_X, b)$. Theorem 1 bounds it:

and, from the numerical solution of (4), gives the values $D_{\tt mse} \approx 0.36$, 0.117, 0.03 and 0.009 at $b = 1$, 2, 3 and 4. These are fractions of the vector's squared length: at 4 bits the reconstruction error is under 1% of it. The constant $\sqrt{3}\pi/2$ comes from the Panter-Dite high-resolution formula, which approximates the optimal error when there are many small cells, applied to the Gaussian limit of $f_X$.

The error falls by a factor of 4 per bit once there are many cells. One more bit doubles the number of centroids, which roughly halves the width of every cell. The error inside a cell is proportional to the square of its width, so halving the width divides the error by 4. At small $b$ the cells are wide and the factor is smaller: with unrounded Lloyd-Max values the error falls by 3.1, 3.4 and 3.6 from one bit to two, two to three, and three to four.

Theorem 3 shows that no quantizer can do much better. For every randomized quantizer that uses $b$ bits per coordinate, there is a unit vector on which its expected error is at least

A counting argument shows where $1/4^b$ comes from. A code of $b \cdot d$ bits can name at most $2^{bd}$ distinct reconstructions. To bring every point of the sphere within distance $\varepsilon$ of one of them takes on the order of $(1/\varepsilon)^{d}$ points, so $2^{bd}$ points reach $\varepsilon \approx 2^{-b}$, and a squared error of $\varepsilon^2 \approx 4^{-b}$. The paper's proof is the rigorous version: the Shannon lower bound from rate-distortion theory gives the floor for a vector drawn uniformly from the sphere, and Yao's minimax principle turns that average-case floor into a worst-case one for any randomized algorithm.

The upper bound is $2.72/4^b$ and the floor is $1/4^b$, so TurboQuant is within a factor of $\sqrt{3}\pi/2 \approx 2.7$ of optimal at every bit-width. The actual values sit closer: at one bit the ratio is $0.363/0.25 = 1.45$, and it rises toward 2.7 as $b$ grows. Figure 4 plots the floor, the guarantee and the achieved values on a log scale, where a constant factor is a constant vertical gap between two parallel lines.

Several pieces of this are high-dimensional limits: the Gaussian shape of $f_X$, the near-independence of coordinates, the $\sqrt{3}\pi/2$ constant (computed on the Gaussian), and the clean $1/4^b$ floor (the exact Shannon bound for the sphere carries a dimension-dependent factor that tends to 1, via Stirling's approximation). At finite $d$ the coordinate density is exactly the Beta of Lemma 1, so "within 2.7×" is a large-$d$ statement. At the dimensions in the experiments (128 per head, 1,536 and 3,072 for embeddings) the paper's measured errors match the predicted ones closely.

Why the MSE quantizer biases inner products

A reconstruction with small squared error gives inner products with small squared error, but they are not unbiased. The one-bit case shows the mechanism. The centroids are $\pm\sqrt{2/(\pi d)}$, so the quantizer stores $\mathrm{sign}(\boldsymbol{\Pi}\mathbf{x})$ and decodes to $\sqrt{2/(\pi d)}\;\boldsymbol{\Pi}^{\!\top}\mathrm{sign}(\boldsymbol{\Pi}\mathbf{x})$. For large $d$, the expected inner product of that reconstruction with a query is

so every inner product comes out about 36% too small on average. The shrinkage comes from the centroid condition: a centroid is the mean of the values in its cell, and that mean lies inside the cell, closer to zero than the values near its outer edge. Replacing each coordinate by its cell's mean keeps the direction of the vector and shrinks its length.

The shrinkage has an exact value at every bit-width. Because each centroid is the conditional mean of its cell, the rounding error is uncorrelated with the centroid, which gives $\mathbb{E}[x\,c] = \mathbb{E}[c^2] = \mathbb{E}[x^2] - \text{MSE}$ for a coordinate $x$ and its centroid $c$. For coordinates scaled to unit variance, the slope of the estimate against the truth is

which is $2/\pi = 0.637$ at one bit, 0.883 at two, 0.965 at three and 0.990 at four. The paper states only the one-bit value and that the bias shrinks as $b$ grows; the formula for $\alpha(b)$ is our derivation from the Lloyd-Max conditions.

A concrete consequence for attention: if every query-key score were shrunk by the same factor 0.64, the softmax would behave as if its temperature had been raised by $1/0.64 \approx 1.57$. Two keys with scores 4 and 2 get attention weights 0.88 and 0.12; scaled to 2.56 and 1.28 they get 0.78 and 0.22. Figure 5 shows the MSE estimates along a line of slope $\alpha(b)$ instead of the diagonal, with the gap growing in proportion to the true inner product.

How the two-stage inner-product quantizer works

TurboQuant removes the bias with a 1-bit estimator that is unbiased by construction: QJL, the Quantized Johnson-Lindenstrauss transform, published in 2024 by two of the same authors with Insu Han. The Johnson-Lindenstrauss lemma says that projecting vectors onto a modest number of random Gaussian directions approximately preserves their lengths and inner products. QJL keeps only the sign of each projection. With a matrix $\mathbf{S} \in \mathbb{R}^{d\times d}$ of i.i.d. $\mathcal{N}(0,1)$ entries, it stores $\mathrm{sign}(\mathbf{S}\mathbf{x})$, one bit per row, and decodes as

Multiplying by $\sqrt{\pi/2} \approx 1.253$ makes the estimate unbiased. For one Gaussian row $\mathbf{s}$ and a unit vector $\mathbf{x}$, write $a = \langle \mathbf{s}, \mathbf{x}\rangle$. The part of $\mathbf{s}$ orthogonal to $\mathbf{x}$ is independent of $a$ and averages to zero, so $\mathbb{E}[\mathrm{sign}(a)\,\mathbf{s}] = \mathbb{E}|a|\,\mathbf{x} = \sqrt{2/\pi}\,\mathbf{x}$. Summing over $d$ rows and multiplying by $\sqrt{\pi/2}/d$ leaves exactly $\mathbf{x}$ on average, so the inner-product estimate is unbiased for every $\mathbf{y}$. Its variance is bounded (Lemma 4 of the paper) by

which is large for a single bit per coordinate: for a unit query at $d = 128$, a standard deviation up to 0.11, comparable to typical inner products between unrelated unit vectors in that dimension.

The MSE quantizer has low variance and a bias; QJL has no bias and high variance. TurboQuant's inner-product quantizer (Algorithm 2) uses both, at a total of $b$ bits per coordinate:

Stage one: run the MSE quantizer at $b-1$ bits and decode it to $\tilde{\mathbf{x}}_{\tt mse}$. The residual $\mathbf{r} = \mathbf{x} - \tilde{\mathbf{x}}_{\tt mse}$ is the part of the vector this stage missed; its expected squared norm is $D_{\tt mse}(b-1)$, 0.117 at $b-1 = 2$.

Stage two: spend the last bit per coordinate on QJL of the residual, $\mathrm{sign}(\mathbf{S}\mathbf{r})$, and store the scalar $\lVert\mathbf{r}\rVert_2$ alongside, because the signs record only the residual's direction.

The estimate of an inner product adds the two parts:

To see that it is unbiased, fix the first stage's output. QJL is unbiased on the residual, so the second term averages to $\langle \mathbf{y}, \mathbf{r}\rangle$, and the sum averages to

whatever the first stage produced; averaging over the first stage then changes nothing. The variance is QJL's, scaled by the squared norm of what it sketches. The residual is small, so the paper bounds the inner-product error by

With $D_{\tt mse}(0) = 1$ (no first stage at $b = 1$, so the residual is $\mathbf{x}$) this gives, for a unit query, $D_{\tt prod} \approx 1.57/d$, 0.56/$d$, 0.18/$d$ and 0.047/$d$ at $b = 1$ to 4, the values Theorem 2 lists. Its closed-form bound is $\sqrt{3}\pi^2/(d \cdot 4^b)$ and the floor from Theorem 3 is $1/(d \cdot 4^b)$, so the inner-product guarantee is within a factor of about 17 of optimal, and the achieved values at $b = 1$ to 4 sit 6 to 12 times above the floor. The inner-product toggle on Figure 4 plots them.

Bit accounting for a 128-dimensional key at $b = 3$: a 2-bit index per coordinate (256 bits), one QJL sign per coordinate (128 bits), the residual norm and the vector's norm. That is the same 384 bits per vector as the 3-bit MSE quantizer, plus one extra float.

# TurboQuant_prod (Algorithm 2): b bits per coordinate, unbiased
S = randn(d, d)                     # i.i.d. N(0,1), fixed and shared

def quant_prod(x):
    idx = quant(x)                  # TurboQuant_mse at b-1 bits
    r = x - dequant(idx)            # residual
    return idx, sign(S @ r), norm(r)

def dequant_prod(idx, qjl, gamma):
    x_mse = dequant(idx)
    x_qjl = sqrt(pi / 2) / d * gamma * (S.T @ qjl)
    return x_mse + x_qjl            # E[<y, result>] = <y, x> for every y

Why put the bits into a coarser MSE stage instead of spending all of them on QJL? QJL with $b \cdot d$ sign bits is also unbiased, and its variance falls only as $1/b$. The two-stage estimator's variance falls as $D_{\tt mse}(b-1)$, by a factor of 3 to 4 per bit. Figure 6 measures all three options on 2,000 random pairs of unit vectors at $d = 128$. The readouts give the mean error (bias) and $d$ times the mean squared error, which is directly comparable with the paper's 1.57, 0.56, 0.18, 0.047.

The readouts show three things. At $b = 1$ the two-stage quantizer is QJL, with $d$ times the squared error near 1.57 when the true inner product is near zero. From $b = 2$ on it is narrower than spending every bit on QJL: about 0.054 against 0.39 at four bits. And the MSE quantizer's histogram is narrower still at two or more bits, but shifted off zero by a bias that grows with the true inner product; at an inner product of 0.3 and one bit its mean error is about −0.11, and at an inner product of 1 and two bits the bias alone makes its squared error more than four times the two-stage quantizer's. The paper's Figure 3 finds the same crossover on DBpedia embeddings: the two-stage quantizer has lower inner-product error at low bit-widths and the MSE quantizer overtakes it as bits increase.

TurboQuant results on KV cache compression and vector search

All experiments run on a single NVIDIA A100 GPU. The first set checks the theory on 100,000 DBpedia entity embeddings of dimension 1,536 with 1,000 queries: the measured MSE and inner-product errors fall between the floor and the guarantee across bit-widths, the MSE quantizer's inner-product bias shrinks with $b$ and grows with the average inner product, and the two-stage quantizer's error spread stays the same as the inner product changes.

KV cache

A decoder keeps every past token's keys and values so that each new token can attend to them; at long context the cache dominates memory (the motivation behind PagedAttention and multi-query attention, which shrink it in other ways). Compressing it fits TurboQuant's two constraints: keys arrive one token at a time, and attention consumes them through inner products. TurboQuant also quantizes the tokens produced during generation, which KIVI and PolarQuant leave in full precision.

On the needle-in-a-haystack test, a sentence is hidden at some depth in a document of 4k to 104k tokens and the model must retrieve it. With Llama-3.1-8B-Instruct and every method held to 25% of the full KV cache memory, the recall scores were:

• Full precision 0.997, TurboQuant 0.997, PolarQuant 0.995.
• KIVI, a scalar quantizer without formal guarantees, 0.981.
• PyramidKV 0.895 and SnapKV 0.858, which evict tokens instead of quantizing them.

The paper attributes the ordering to theoretical guarantees: PolarQuant and TurboQuant have them, and KIVI and the eviction methods do not.

On LongBench-E, a set of long-context tasks (question answering over one or several documents, summarization, few-shot learning, synthetic retrieval, code completion), the paper reports Table 1:

At 3.5 bits TurboQuant matches the full cache's average exactly (50.06). At 2.5 bits it loses 0.62 points on Llama and 0.27 on Ministral, with the largest single drop on summarization, and still beats KIVI at 3 bits by almost a point. The fractional bit-widths come from an outlier split, the same observation SmoothQuant builds on: a few channels of each head carry much larger values. TurboQuant divides each head's 128 channels into an outlier set and the rest, and runs two independent instances at different bit-widths; 64 channels at 3 bits plus 64 at 2 bits average 2.5. The paper reports compressing the quantized vectors by at least 4.5× (16/3.5 ≈ 4.6, and 16/2.5 = 6.4).

Nearest-neighbour search

The search experiments use DBpedia embeddings from OpenAI's text-embedding-3-large at $d = 1536$ and $3072$, and GloVe at $d = 200$, with 100,000 database vectors each. The baselines are product quantization (PQ), which learns its codebooks with k-means on the dataset, and RaBitQ, a quantizer with theoretical guarantees that projects a uniform grid onto the sphere (the multi-bit version, which extends the 1-bit RaBitQ of 2024). The metric is recall@1@k: how often the true best match is among the top $k$ results by estimated inner product. TurboQuant has higher recall than both at 2 and 4 bits on all three datasets, although PQ was trained and evaluated on the same vectors, which favours it.

Figure 7 plots the encoding (indexing) time from Table 2 at 4 bits. PQ spends 37 to 494 seconds running k-means. RaBitQ spends 597 to 3,957 seconds; it has no GPU implementation, so in this comparison it runs on the CPU. TurboQuant takes 0.0007 to 0.0021 seconds, a rotation and a table lookup on the GPU, between about 53,000 and 1.9 million times faster.

What the guarantees cover

The theorems bound expected error over the random rotation (and the random QJL matrix), for the worst unit input. A particular rotation can do slightly better or worse on a particular vector, as the new-rotation button in Figure 3 shows. The constants are high-dimensional limits, and the 2.7 factor applies to squared error; the inner-product guarantee is within about 17 of its floor.

The KV-cache results use an outlier split that sits outside the theory: the theorems cover a single quantizer at an integer bit-width on unit vectors, and choosing which channels are outliers is a separate step that the paper does not describe. The paper reports two models, Llama-3.1-8B-Instruct and Ministral-7B-Instruct, and one GPU type. It compares search recall in figures only, and the timing comparison with RaBitQ is GPU against CPU. The paper does not report end-to-end decoding speed for the KV-cache experiments.