NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Fit a small network to photos of one scene, then render that scene from any new camera angle.

NeRF stores a 3D scene as a function, computed by a neural network of about 5 MB, that returns a color and a density for any point in space and any viewing direction. A new view is rendered by sampling that function along each camera ray and compositing the samples, and the network is trained by making its renders match the photos.

From 100 photos of an object with known camera positions, NeRF renders unseen views at 31.01 dB PSNR (peak signal-to-noise ratio, a log measure of pixel error; higher is better), 4.96 dB above the best earlier method on the paper's path-traced synthetic benchmark. Each new scene costs one to two days of training on a GPU.

The task is view synthesis. You have a few dozen photos of a scene, each taken from a known camera position, and you want an image from a camera position nobody photographed, for example every frame of a smooth fly-around video. The older recipes build an explicit model first and then render it. A triangle mesh needs the right topology before it can be optimized, and real scenes rarely come with one. A voxel grid fine enough to look crisp grows with the cube of its resolution; the Neural Volumes baseline in the paper uses a 128³ grid, which the authors say keeps it from representing fine detail at high resolution. Local Light Field Fusion, an earlier method by the same first author, predicts a stack of semi-transparent image layers for every input photo and needs over 15 GB for one of the paper's synthetic scenes.

NeRF, from Ben Mildenhall and colleagues at UC Berkeley, Google Research and UC San Diego in March 2020, keeps no explicit model. The scene is a function from a 3D point and a viewing direction to a color and a density, and that function is a fully-connected network. Rendering a pixel means querying the network at points along the pixel's camera ray and adding up what comes back with the volume-rendering rule graphics has used since the 1980s. Every step of that render is differentiable, so the network is trained by rendering pixels of the training photos, comparing them with the real pixels, and descending on the squared error.

Five ideas carry the paper, and the sections follow them in order: the scene as a queryable function with density tied to position only; volume rendering, which turns that function into pixels; training on photometric error alone; a positional encoding of the inputs, without which the renders come out blurred; and coarse-to-fine sampling, which puts network queries where the surfaces are. Together they set a new state of the art for photorealistic novel views of real, complex scenes from ordinary photos.

What a NeRF stores

A NeRF is one continuous function $F_\Theta$ with learnable weights $\Theta$. It takes a 3D location $\mathbf{x} = (x, y, z)$ and a viewing direction $\mathbf{d}$, and returns an emitted color $\mathbf{c} = (r, g, b)$ and a volume density $\sigma$:

The paper calls the input 5D: three numbers for the position and two angles $(\theta, \phi)$ for a direction on the sphere. In practice the direction is passed as a 3D unit vector, so the network reads six numbers. Nothing is stored on a grid. Every point in space has a color and a density, and they are whatever the weights compute when that point is queried. Density says how much material is at a point (zero in empty air, large inside a solid object), and color is the light that point sends toward the camera.

The network is a multilayer perceptron, a stack of fully-connected layers with no convolutions and no attention. Its weights, for the two networks the method trains, take about 5 MB, which is less than the photos it was fit to.

A NeRF is fit from scratch to a single scene. There is no training set of many scenes and no generalization to a new one: the optimization overfits one scene's photos on purpose, and the weights describe that scene and nothing else, the way a compressed file holds one specific song. A new object means a new optimization of one to two days on a V100 GPU. What comes out is also not a mesh or a point cloud. The geometry exists only as the pattern of densities the network reports when queried, so a mesh for a game engine needs an extra step that samples the densities and extracts a surface.

Density from position, color from direction

The two outputs depend on different inputs. Density $\sigma$ is a function of the position $\mathbf{x}$ alone. Color $\mathbf{c}$ is a function of the position and the viewing direction $\mathbf{d}$.

Why not let both depend on everything? Density is geometry, and the corner of a table is in the same place from every camera. If density could depend on direction, the network could give each training photo its own shape, fit every photo exactly, and produce nonsense from any camera in between. With density tied to position, one density field has to explain all the photos at once, which is what pushes it toward the real geometry; the paper gives this multiview consistency as the reason for the split.

Color gets the direction because real appearance changes with viewpoint. A glossy surface shows a specular highlight that moves as you walk past it; varnished wood and metal look different head-on than at a grazing angle. The output is radiance, the light a point sends out in a particular direction, and a color that depends only on position can show a highlight only as a blur averaged over all the photos.

Orbit the viewing angle in Figure 1. The density bar does not move. The color swatch brightens as your view lines up with the mirror reflection of the light and returns to the surface's dull amber as you move away.

The architecture enforces the split by wiring. The encoded position runs through eight fully-connected ReLU layers of 256 units, with a skip connection that concatenates the encoded input onto the fifth layer's output, a design taken from DeepSDF (an earlier network that maps coordinates to a shape's signed distance). After the eighth layer, one linear layer outputs $\sigma$ (passed through a ReLU so it cannot be negative) and another outputs a 256-number feature vector. Only then is the encoded direction concatenated onto that feature, passed through one more ReLU layer of 128 units, and turned into RGB by a sigmoid. Direction never reaches the layers that compute $\sigma$, and it passes through only one hidden layer, so its effect is limited to adjusting the color of a surface the trunk has already placed.

The shapes, with the released code's encodings (63 numbers for a position, 27 for a direction; the encoding section explains the sizes): the first layer maps 63 to 256, the layer after the skip maps 256 + 63 = 319 to 256, and the color layer maps 256 + 27 = 283 to 128. Counting weights and biases gives 595,844 parameters per network. NeRF trains two such networks (the coarse and fine networks of a later section), 1,191,688 parameters in all, which is 4.77 MB in 32-bit floats and matches the paper's "5 MB".

How NeRF renders a pixel

A function that returns color and density at points in space is not yet an image. Each pixel defines a ray that leaves the camera center $\mathbf{o}$ in a direction $\mathbf{d}$ fixed by the pixel's position and the camera's calibration:

The parameter $t$ measures distance along the ray. To color the pixel, NeRF samples points along the ray, queries the network at each, and combines the answers. An 800 × 800 image has 640,000 pixels, so 640,000 rays. In Figure 2 a camera on the left casts one ray per pixel of a column of the image into a scene with a teal blob and an amber blob. Pick a pixel and its ray lights up with sample points, each tinted by the color returned there and sized by the density; their composite is the pixel on the image plane. A ray that crosses a blob takes its color; a ray through the gap reaches the dark background.

The rule for combining samples is the volume-rendering integral. The expected color of a ray between a near bound $t_n$ and a far bound $t_f$ is

Read it from the camera outward. At each depth $t$ the material emits light of color $\mathbf{c}$ in proportion to its density, the product $\sigma\,\mathbf{c}$. That light is dimmed by $T(t)$, the transmittance: the fraction of light that crosses the stretch from $t_n$ to $t$ without being absorbed. Transmittance starts at 1 and falls exponentially in the total density crossed so far, the same law that governs light through fog or coffee. Dense material near the front drives $T$ toward zero, so nothing behind it reaches the pixel; occlusion comes out of the integral with no separate visibility test.

The paper gives two readings of the same formula. The physical one: $\sigma$ is an extinction coefficient, the rate at which material absorbs light per unit length, so it has units of one over length and can be any non-negative number. The probabilistic one: $\sigma(\mathbf{x})$ is the differential probability that the ray stops at a particle at $\mathbf{x}$, $T(t)$ is the probability it travels to $t$ without stopping, $T\sigma$ is the density of where it stops, and $C(\mathbf{r})$ is the expected color at the stopping point. The second reading is exact only if the ray is certain to stop somewhere. Over a finite range a fraction $T(t_f)$ of the light passes all the way through, and that leftover lets the background show.

Equation (1) is Nelson Max's 1995 absorption-plus-emission model with the background term dropped. It has no scattering: light is never bounced from one point onto another, so there are no simulated soft shadows or color bleeding. Each point emits a fixed radiance that already includes the lighting at capture time, and the view-dependent color is a learned stand-in for how the real scene looked from each direction. This is also why a NeRF cannot be relit; the equation has no light source to move.

From the integral to a sum: density becomes opacity

The integral cannot be evaluated exactly, so NeRF estimates it from samples. Split the range $[t_n, t_f]$ into $N$ equal bins and draw one sample uniformly inside each (stratified sampling):

Why random positions instead of a fixed grid of depths? With a fixed grid the network would only ever be queried at the same $N$ depths on every ray, and nothing would constrain it in between; the paper notes that deterministic quadrature would limit the representation's resolution. Redrawing the samples at every training step means the network is queried at a continuous spread of depths over the course of training, while each render still costs only $N$ queries.

With samples $t_1 < \dots < t_N$, let $\delta_i = t_{i+1} - t_i$ be the gap to the next sample. Over a stretch of length $\delta_i$ with constant density $\sigma_i$, transmittance falls from 1 to $e^{-\sigma_i \delta_i}$, so the stretch absorbs a fraction $\alpha_i = 1 - e^{-\sigma_i\delta_i}$, the opacity of the segment. Light reaching sample $i$ has survived every segment before it, which gives the quadrature rule NeRF takes from Max:

The paper writes $T_i$ as $\exp(-\sum_{j<i}\sigma_j\delta_j)$, which is the same product. The product stops at $i-1$: sample $i$ is dimmed by the segments in front of it, and its own opacity $\alpha_i$ appears once, as the share of the arriving light it absorbs and re-emits as its color. The product $w_i = T_i\alpha_i$ is the render weight of sample $i$, how much of the pixel comes from that sample. The sum is front-to-back alpha compositing, the "over" operator used to stack semi-transparent layers in an image editor, with each layer's alpha computed from a density and a length.

For a worked example, take a synthetic scene, where the released code renders rays from $t_n = 2$ to $t_f = 6$, and use 64 samples, so $\delta = 4/64 = 0.0625$. Four consecutive samples report densities 0, 20, 20 and 5: empty air, two samples inside a solid teal surface, and one in a thin amber layer behind it.

The pixel is 0.917 parts teal and 0.022 parts amber. The weights sum to 0.940, and the remaining 0.060 of the light passes through to whatever lies behind. In general

because each weight is the drop in transmittance across its segment, so the sum telescopes to one minus the light that survives the whole ray. The sum equals 1 only if the ray is fully absorbed. The released code uses the leftover directly: for the synthetic scenes, which are composited on white, it adds $1 - \sum_i w_i$ times white to the pixel.

Why the exponential and not the simpler $\alpha_i = \sigma_i\delta_i$? That is only the first term of the exponential's expansion, accurate when $\sigma_i\delta_i$ is small. For the teal samples above it gives 1.25, an impossible opacity; transmittance after the first teal sample would be $1 - 1.25 = -0.25$, and the next sample would get a negative weight. The exponential stays in $[0, 1)$ for any density and any gap, and a solid surface sampled with fewer, wider segments still comes out solid.

Figure 3 draws every quantity of Eq. (3) for one ray through a thin teal surface and a semi-transparent amber one, in front of a light background. Drop the sample count to 4 and watch $\Sigma w$ and the pixel drift from the exact integral as samples miss the thin surface; switch to $\alpha = \sigma\delta$ to see the negative weights; set the front density to 0 and the amber layer lets some background through.

The whole render is a few array operations, each differentiable, so a gradient flows from a pixel's error back to every $\sigma_i$ and $\mathbf{c}_i$ the network produced along that ray:

# render one camera ray r(t) = o + t*d into a pixel color (Eq. 2, 3)
ts    = stratified(t_near, t_far, N=64)    # one uniform draw per bin
pts   = o + ts[:, None] * d                # (64, 3) points on the ray
rgb, sigma = F(gamma(pts), gamma(d))       # (64, 3) colors, (64,) densities
delta = diff(ts, append=t_far)             # gap to the next sample
alpha = 1 - exp(-sigma * delta)            # opacity of each segment
T     = cumprod(1 - alpha, exclusive=True) # light left on arrival: T_1 = 1
w     = T * alpha                          # render weight per sample
C     = sum(w[:, None] * rgb, axis=0)      # composite front to back
C     = C + (1 - sum(w)) * background      # leftover light shows the background

How NeRF is trained from photos

Everything above is a differentiable function from network weights to pixel colors, so fitting the network is ordinary gradient descent. Each step picks a batch $\mathcal{R}$ of 4096 random pixels from across all training photos, renders their rays, and scores the squared error against the true colors. NeRF renders every ray twice, with a coarse and a fine network (the subscripts $c$ and $f$, explained two sections on), and supervises both renders:

The loss has no 3D supervision: no depth, no silhouettes, no mask, nothing about geometry. The only signal is that a pixel came out the wrong color. Geometry still emerges because one density field has to produce every photo. Suppose the network puts density at the wrong depth along some ray of camera A. That ray's pixel can still come out right, but rays from camera B cross the same misplaced density at other pixels, where it blocks or tints the wrong things, and their errors push it away. A density arrangement that renders every training photo correctly has to sit where the surfaces really are, provided enough cameras see each point from different angles.

Figure 4 runs this on a two-dimensional scene you can train in the browser. To make it fast, the network is replaced by a 28 × 28 grid of densities and colors; the rest is NeRF's: stratified samples (Eq. 2), the quadrature of Eq. (3), the squared error of Eq. (6), Adam. Each camera takes a one-row, 40-pixel photo of the true scene. With one camera, the fit matches that camera's photo to within a fraction of a gray level while the density smears along its rays, and the held-out camera, whose photo never enters the loss, sees the wrong image. Add cameras and the density collapses onto the disc and the square.

At $K = 1$ the training camera reaches about 57 dB while the held-out camera scores under 10 dB: the optimization has reproduced its one photo exactly and learned nothing about depth. Averaged over 16 held-out viewpoints, PSNR rises from 11 dB at one camera to about 22 dB at eight or more. The same pattern in the paper: with 100 training images NeRF scores 31.01 dB on held-out views, with 50 it scores 29.79, and with 25, 27.78.

Adam runs with a learning rate that decays exponentially from $5 \times 10^{-4}$ to $5 \times 10^{-5}$ (and $\epsilon = 10^{-7}$), and a scene converges in 100,000 to 300,000 steps, one to two days on a single V100. The sketch below is the loop, with the coarse-to-fine sampling of a later section already in place:

# fit ONE scene (Eq. 6). Nothing but photos and their camera poses goes in.
for step in range(300_000):                    # 100-300k steps, 1-2 days, one V100
    o, d, px = sample_rays(images, poses, 4096)  # 4096 random pixels, all images
    C_c, w_c = render(coarse, o, d, N=64)        # stratified pass (Eq. 2, 3)
    t_f = sample_pdf(w_c, N=128)                 # extra depths where w_c is big
    C_f, _ = render(fine, o, d, union(t_c, t_f)) # 192 sorted samples
    loss = sum_sq(C_c - px) + sum_sq(C_f - px)   # both renders are supervised
    loss.backward(); adam.step()                 # lr 5e-4 decaying to 5e-5

Only the photos supervise the network, and the coarse network's output enters the loss directly. In the released code the fine sample depths are wrapped in tf.stop_gradient, so no gradient flows through where the fine samples were placed; the coarse network learns to place them well only through its own loss term.

Why a plain MLP blurs: positional encoding

Trained as described so far, on raw $(x, y, z)$ and direction, NeRF renders soft, washed-out images with no crisp edges or fine texture. Capacity is not the limit; an MLP of this size can represent such a scene. Rahaman and colleagues showed in 2019 that networks trained by gradient descent fit the low-frequency parts of a function first and the high-frequency parts very slowly, which they called spectral bias. An edge or a fine texture is high-frequency content, so within a realistic training budget the network never gets to it.

NeRF changes the inputs instead of the network. It writes $F_\Theta = F'_\Theta \circ \gamma$: a fixed encoding $\gamma$ with no learned weights, followed by the learned MLP $F'_\Theta$. Each coordinate $p$, normalized to $[-1, 1]$, becomes $2L$ numbers:

The encoding is applied to each of the three position coordinates with $L = 10$ and to each of the three direction components with $L = 4$, so the paper's network reads $3 \times 20 = 60$ numbers for a position and $3 \times 8 = 24$ for a direction. Band $k$ completes $2^k$ cycles across $[-1, 1]$, so the top position band has 512.

The pair $(\sin 2^k\pi p,\ \cos 2^k\pi p)$ is a point on a unit circle at angle $2^k\pi p$. Two positions that differ by $\Delta$ sit $2\,\lvert\sin(2^{k-1}\pi\Delta)\rvert$ apart on band $k$'s circle, regardless of where they are. For two points 0.01 apart, about 0.5% of the scene's width, that distance is 0.031 on band 0, 0.25 on band 3, 0.96 on band 5 and 1.97 on band 9. In the raw coordinate they differ by 0.01; in the encoded input they differ by order 1 on the high bands, so the first layer can assign them very different densities or colors, which is what a sharp edge needs.

The paper offers the encoding as an empirical fix and points to spectral bias for the reason. A theory came a few months later in Fourier Features Let Networks Learn High Frequency Functions by Tancik and colleagues (overlapping authors), which analyzes the encoding through the network's neural tangent kernel. The sinusoid form echoes the positional encoding of the Transformer, which used it to give tokens an order rather than to lift continuous coordinates.

Figure 5 trains a real network in your browser to show both effects. The target is a 1D signal with detail at 1, 3, 7, 15 and 31 cycles across $[-1, 1]$, sampled at 200 points; a ReLU MLP with two hidden layers of 32 units is fit to the samples with Adam for 1600 steps, fed $p$ plus $L$ bands. At $L = 0$ it fits the slow trend and stays smooth (mean squared error 0.051). Each band lets it reach finer detail, and by $L = 4$ or 5 it matches the target (0.0002). At $L = 8$ the top band has 128 cycles, more than 200 samples can pin down (they resolve at most 100): the fit passes through every training sample and oscillates between them, which the error measured between samples exposes (0.047). At $L = 7$ the top band has 64 cycles, inside that limit, yet the error between samples still rises to 0.010, most likely because the network combines bands into components above 100 cycles that the samples do not constrain.

NeRF's own frequency ablation shows a milder version of the plateau: $L = 5$ scores 30.59 dB, $L = 10$ 31.01 and $L = 15$ 30.81. The paper attributes this to $2^L$ passing the highest frequency present in the sampled input images, roughly 1024 in their data; past that point extra bands have nothing left to fit. Removing the encoding entirely drops the full model from 31.01 to 28.77 dB.

The released code differs from Eq. (4) in two ways. It prepends the raw coordinate to the bands, so a position is encoded as $3 + 6L = 63$ numbers and a direction as 27, and it uses frequencies $2^k$ without the factor $\pi$. Since the inputs lie in $[-1, 1]$, dropping $\pi$ only shifts which band carries which frequency. Figure 5 keeps the raw input, as the code does, so that $L = 0$ still has something to read.

Coarse and fine sampling

Uniformly spaced samples spend most of their queries where they do nothing. A ray crosses mostly empty air, and everything behind the first opaque surface has $T_i \approx 0$, so the weight $w_i = T_i\alpha_i$ of most samples is close to zero while each still costs a full network evaluation. The samples that matter sit in the thin shell where the ray meets a visible surface, and that shell is unknown until the ray has been rendered once.

NeRF renders each ray twice. A coarse network is queried at $N_c = 64$ stratified samples and composited as in Eq. (3), giving weights $w_i$. Normalizing them,

turns them into a piecewise-constant probability density along the ray, high where the coarse pass found something. The division is needed because the raw weights sum to at most 1, as above. NeRF draws $N_f = 128$ more depths from that density by inverse-transform sampling: build the running total of the normalized weights, draw uniform numbers in $[0, 1]$, and map each to the depth where the running total crosses it, which puts more samples where the weights climb fastest. A second, fine network is then evaluated at all 64 + 128 = 192 depths, sorted, and its composite is the final pixel. Both networks are trained together; the coarse network's term in Eq. (6) is what keeps its weights useful for placing the fine samples.

# inverse-transform sampling from the coarse weights (Sec. 5.2)
p   = w_c / sum(w_c)                  # normalize: raw weights sum to <= 1
cdf = concat([0], cumsum(p))          # running total, from 0 up to 1
u   = uniform(0, 1, size=128)         # 128 uniform draws
i   = searchsorted(cdf, u)            # bin where each draw crosses the cdf
t_f = edges[i-1] + (u - cdf[i-1]) / p[i-1] * (edges[i] - edges[i-1])
t   = sort(concat([t_c, t_f]))        # 64 + 128 = 192 depths for the fine net

Drag the surface in Figure 6. The coarse samples cover the ray uniformly and only those near the surface carry weight; the normalized weights form the violet density; the fine samples drawn from it crowd the surface and skip the air on either side. Move the surface and the fine cluster follows it, because its positions come from the coarse result.

This is not Monte Carlo importance sampling, although the drawing step looks the same. An importance sampler divides each sample's contribution by the probability of drawing it and averages the results as independent estimates of the integral. NeRF does neither: the drawn depths become extra nodes of the same compositing sum of Eq. (3), a non-uniform grid that is dense where the surface is. The paper describes it as using the samples "as a nonuniform discretization of the whole integration domain".

The ablation compares this against one network with 256 stratified samples, the same 256 queries per ray. The coarse-to-fine model scores 31.01 dB and the single network 30.06. A rendered pixel costs 64 coarse plus 192 fine queries, 256 in all; at 640,000 rays per synthetic image and 762,000 per real one, that is 150 to 200 million network evaluations per image, about 30 seconds per frame on a V100.

Real photos: camera poses and unbounded depth

The synthetic scenes come with exact camera poses and are scaled into a cube of side 2 centered at the origin. Real photos need two more steps. First, NeRF does not estimate camera positions itself: poses, intrinsics and scene bounds for the real scenes come from COLMAP, an off-the-shelf structure-from-motion program that matches features across photos and solves for where each photo was taken.

Second, a real scene can extend to the horizon, which breaks the finite $[t_n, t_f]$ range the integral needs. For the forward-facing captures (a phone pointed roughly the same way while the photographer moves a little) NeRF reparameterizes space into normalized device coordinates, the projective space a rasterizer uses. Depth is replaced by disparity, one over depth, which runs from a finite value at the near plane down to zero at infinity, and the transform maps that range to $[-1, 1]$. Uniform samples in this space are uniform in disparity, so they are dense near the camera and sparse far away, and a mountain at infinity lands on the far plane instead of at an infinite $t$.

For real scenes only, training adds zero-mean, unit-variance Gaussian noise to the network's raw density output before the ReLU; the paper reports this slightly improves novel views. The real dataset is eight scenes shot with a handheld phone (five from the LLFF paper, three new), 20 to 62 photos each at 1008 × 756, with one photo in eight held out for testing.

Results and ablations

The main benchmark, Realistic Synthetic 360°, is eight objects with complicated geometry and non-Lambertian (shiny, view-dependent) materials, path-traced in Blender, with 100 training views and 200 test views at 800 × 800. NeRF reaches 31.01 dB PSNR. The baselines, each run with the same input views: Neural Volumes 26.05, Local Light Field Fusion 24.88, and Scene Representation Networks (SRN, which marches each ray with a recurrent network and keeps a single depth and color per ray) 22.26. PSNR is a log scale of mean squared pixel error, and every 3 dB roughly halves the error. The 4.96 dB gap to Neural Volumes is a factor of $10^{0.496} \approx 3.1$ in mean squared error: on the paper's figures, the difference between a clean specular highlight and a gray smear. On the four simpler Lambertian objects of the Diffuse Synthetic 360° set, NeRF reaches 40.15 dB.

On the real forward-facing scenes NeRF scores 26.50 dB PSNR and 0.811 SSIM (structural similarity, from 0 to 1, higher is better) against LLFF's 24.13 and 0.798, but LLFF has the better LPIPS, a learned perceptual distance where lower is better: 0.212 against NeRF's 0.250, the only metric in Table 1 on which NeRF trails a baseline. NeRF also stays consistent from view to view, while LLFF blends a separate representation per input view, which the paper calls perceptually distracting in its supplementary video.

NeRF's weights take 5 MB, about 3000 times less than the over 15 GB LLFF stores for one Realistic Synthetic scene, and less than the input images. LLFF, the only baseline trained once on a large dataset and then applied to new scenes, processes a new scene in under 10 minutes; every method that trains per scene, NeRF included, takes at least 12 hours.

The ablation in Figure 7 removes one piece at a time on the Realistic Synthetic scenes. Without view dependence PSNR falls from 31.01 to 27.66, without the positional encoding to 28.77, without hierarchical sampling to 30.06, and with all three removed to 26.67. The second view covers training data and frequency bands: 50 images give 29.79, 25 images 27.78, and $L = 5$ or $L = 15$ both land below $L = 10$. The paper notes that even the 25-image model beats Neural Volumes, SRN and LLFF trained on 100 images.

What NeRF cannot do

Rendering is slow: 256 network queries per pixel, about 30 seconds per frame. Fitting is slow: one to two days per scene with nothing reused from earlier scenes. The lighting is baked into the emitted color, so a NeRF cannot be relit and yields no materials. The geometry is implicit, so a mesh for another tool requires extracting a surface from sampled densities. Camera poses must be known in advance. The paper also lists interpretability: a voxel grid or mesh lets you reason about expected quality and failure modes, and it is unclear how to do that for a scene stored in network weights.

Each limit became a line of follow-up work: faster training with learned feature grids, scenes generalized across with image-conditioned networks, relightable variants that separate lighting from materials. The best-known rendering fix, 3D Gaussian Splatting, keeps the front-to-back compositing of Eq. (3) and replaces the network with millions of explicit 3D Gaussians that are projected and blended in real time.

Implementation notes

The official TensorFlow code, bmild/nerf, differs from the paper text in a few places that matter when matching one against the other.

• The encoding prepends the raw coordinate and drops the $\pi$ (63 and 27 inputs, frequencies $2^k$), as the encoding section describes.
• The loss is tf.reduce_mean of squared errors, a mean over the batch, where Eq. (6) prints a sum. The two differ by a constant factor that rescales the effective learning rate.
• The last sample on each ray gets a segment length of $10^{10}$, so any positive density there absorbs everything that is left.
• Transmittance is an exclusive cumprod of 1 - alpha (plus 1e-10 for stability): exclusive means it gives $T_1 = 1$ and stops at $i - 1$, as in Eq. (3).
• Synthetic scenes use white_bkgd: training images are composited on white, and each render adds $1 - \sum_i w_i$ of white.
• The example configs in the repository, such as config_lego.txt, use 64 fine samples, 1024 rays per batch and half-resolution images, a lighter setting than the paper's 128 and 4096 at full resolution.