GCN: Semi-Supervised Classification with Graph Convolutional Networks

Mix each node's features with its neighbors', and 140 labels are enough to classify every node of a 2,708-node graph.

A graph convolutional network multiplies the node features by a normalized adjacency matrix before each weight matrix. Kipf and Welling derive that matrix from spectral graph theory, and two such layers beat every baseline they tested on four node-classification benchmarks.

You have a citation network: 2,708 papers, 5,429 citation links, and a subject label on 20 papers per subject. Predict the subject of the other 2,568.

A paper's own text is one source of evidence for its subject. Its citations are another: papers mostly cite papers in the same field. A classifier that reads only the text throws the second source away, and with 140 labels it has little to learn from. Kipf and Welling build a neural network that takes both inputs, the feature vector on every node and the adjacency matrix of the graph, and train it with labels on only a few nodes.¹

Each layer of the network replaces every node's feature vector with a weighted sum of its own vector and its neighbors', multiplies the result by a learned weight matrix, and applies a ReLU. Two of those layers and a softmax score 81.5% on Cora, ahead of every method the paper compares against. The rest of this page explains, in order: the older approach the paper replaces (a smoothness penalty in the loss); the layer itself, with a worked example; where its particular weights come from (graph frequencies, a Chebyshev filter, and two approximations); the training setup; the results; why the model stays at two layers; and a second reading of the layer as a trainable version of the Weisfeiler-Lehman graph test.

The task: label a graph from a few nodes

A graph has $N$ nodes. Each node carries a feature vector, and stacking them gives a matrix $X\in\mathbb{R}^{N\times C}$ with $C$ features per node. On Cora a node is a paper and its feature vector is a 0/1 bag-of-words over a 1,433-word vocabulary, so $X$ is $2708\times 1433$. The edges go in the adjacency matrix $A\in\mathbb{R}^{N\times N}$, with $A_{ij}=1$ when nodes $i$ and $j$ are linked and 0 otherwise (the method also accepts positive edge weights in place of the 1s). The graphs here are undirected, so $A$ is symmetric. A small set of nodes has a class label; the job is to predict the class of every other node.

The setting is transductive: the whole graph is available during training, including every edge and the feature vectors of the unlabeled nodes. Only their labels are hidden, and the test nodes are nodes of this same graph. A transductive model can use an unlabeled neighbor's features when it predicts a labeled node, which is how the method spreads 140 labels across 2,708 nodes. It also means the trained model is tied to this graph; producing embeddings for nodes that were not present at training time is the job of a later method, GraphSAGE.⁵

The paper uses four datasets with the splits of the Planetoid benchmark (Yang, Cohen and Salakhutdinov, 2016). Cora has 2,708 nodes, 5,429 edges, 7 classes and 1,433 features, and trains on 20 labels per class: 140 labeled nodes, a label rate of 0.052. Citeseer (3,327 nodes, 6 classes) and Pubmed are citation networks of the same kind. NELL is a knowledge graph trained with a single labeled example per class, a label rate of 0.001. Every citation dataset is scored on 1,000 held-out labeled nodes, and a further 500 labeled nodes serve as a validation set for choosing hyperparameters, never for training.

Laplacian regularization, the approach GCN replaces

Before this paper, the standard way to use the graph in semi-supervised learning was a penalty term in the loss that makes linked nodes receive similar predictions:

$\mathcal{L}_0$ is the supervised loss on the labeled nodes, $f$ is any classifier applied to one node's features at a time, and $\lambda$ weights the penalty. The penalty adds, for every edge, the squared difference between the two endpoints' predictions. If two linked papers get a predicted probability of 0.9 and 0.2 for the same subject, that edge contributes $(0.9-0.2)^2=0.49$, and gradient descent pushes the two predictions toward each other. The right-hand form writes the same sum with $\Delta = D - A$, the unnormalized graph Laplacian, where $D$ is the diagonal degree matrix ($D_{ii}$ is the number of edges at node $i$). Row $i$ of $\Delta f$ equals $D_{ii}$ times the gap between $f_i$ and the mean of its neighbors' values, so the quadratic form $f^\top\Delta f$ measures total disagreement along edges.²

Kipf and Welling object to the assumption inside (1): it treats every edge as a statement that the two nodes share a label. In their words this "might restrict modeling capacity, as graph edges need not necessarily encode node similarity"; a paper can cite work from another field. Their alternative removes the penalty and puts the graph inside the model. The classifier becomes a function $f(X,A)$ of the features and the adjacency matrix together, and training minimizes only $\mathcal{L}_0$ on the labeled nodes. How much a neighbor's features should count is then decided by learned weights, and because a labeled node's output is computed from its neighbors' features, the gradient of $\mathcal{L}_0$ also passes through those neighbors. The model section below walks that gradient step by step, after the layer itself and its derivation.

How a GCN layer works

One layer maps the feature matrix $H^{(l)}$ to the next one:

$H^{(l)}$ has one row per node, and the first layer takes $H^{(0)}=X$. $\tilde{A}=A+I_N$ is the adjacency matrix with a self-loop added at every node, and $\tilde{D}$ is its diagonal degree matrix, so $\tilde{d}_i$ is node $i$'s number of neighbors plus one. Write $\hat{A}$ for the product $\tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}$. Reading (2) from the inside out: $\hat{A}H^{(l)}$ replaces each node's row by a weighted sum of the rows of its neighbors and itself; multiplying by the weight matrix $W^{(l)}$ mixes the feature channels, the same matrix for every node; and $\sigma$ is an elementwise nonlinearity, a ReLU in the paper. The only learned parameters are the $W^{(l)}$; the graph enters only through the fixed matrix $\hat{A}$.

The weight that node $j$'s row gets in node $i$'s sum is the matrix entry, where $\mathcal{N}_i$ is the set of node $i$'s neighbors:

A worked example on the 11-node graph of Figure 1. Node 4 is the bridge between two groups. It has three neighbors (nodes 2, 6 and 10), so $\tilde{d}_4=4$; nodes 2 and 6 have five neighbors each ($\tilde{d}=6$) and node 10 has three ($\tilde{d}=4$). With one scalar feature per node, valued 0.10 on node 4, 0.85 on node 2, −0.70 on node 6 and −0.40 on node 10, node 4's new value is

The four weights are 0.25, 0.204, 0.204 and 0.25, which sum to 0.908, so this is a weighted sum and not an average. A neighbor with many edges of its own gets a smaller weight: nodes 2 and 6 each count for 0.204 against node 10's 0.25. Splitting the normalization as a square root on each side means an edge between two high-degree nodes is scaled down from both ends. The plain row average would be $\tilde{D}^{-1}\tilde{A}$, with weights $1/\tilde{d}_i$ that do sum to one. The paper does not discuss it; the symmetric form is what the spectral derivation in the next two sections produces.

In Figure 1, tap any node. Its edges light up, and the readout lists each contribution: the weight, the neighbor's value, and the sum that becomes the node's new feature. Tapping a node inside a group gives a value close to that group's color; the bridge node lands between them.

Two questions remain about (2): why the paper calls it a convolution, and where the $1/\sqrt{\tilde{d}_i\tilde{d}_j}$ weights and the self-loop come from. The paper answers both by starting from the definition of convolution in signal processing and approximating it.

Graph frequencies and spectral filters

On an image, a 3×3 convolution kernel applies the same nine weights at every pixel, which works because every pixel has the same eight neighbors in the same arrangement. Nodes in a graph have different numbers of neighbors and no ordering among them, so there is no direct analogue of sliding a kernel. Signal processing offers a second definition that does carry over: convolution is multiplication in the frequency domain. A graph needs a set of frequencies for that, and the Laplacian provides them. The paper uses the normalized Laplacian, a different matrix from the $\Delta=D-A$ of (1):

$L$ is symmetric, so it has $N$ orthonormal eigenvectors (the columns of $U$) and real eigenvalues (the diagonal of $\Lambda$), which for this normalization lie between 0 and 2. Each eigenvector assigns one number to every node. An eigenvector with a small eigenvalue changes little across edges; the one with eigenvalue 0 is proportional to $\sqrt{d_i}$ at node $i$, the smoothest signal the graph has. Eigenvectors with large eigenvalues change sign across most edges. The eigenvalue plays the role of a frequency, and the coefficients $U^\top x$ of a node signal $x$ in this basis are its graph Fourier transform. A spectral filter scales each frequency by its own factor and transforms back:

Figure 2 colors the same 11-node graph by one eigenvector at a time. Step the slider from $k=0$ upward. Eigenvector 1 (the Fiedler vector) is already positive on one group and negative on the other at an eigenvalue of 0.08, so the split between the two communities is a low-frequency pattern, and the highest eigenvectors alternate sign from node to node. The teal curve is the response of the operator from equation (7) below, halved: $g(\lambda)=1-\lambda/2$. It passes low frequencies at nearly full gain and suppresses high ones.

Equation (3) is exact and too expensive to use. Multiplying by the dense matrix $U$ costs $O(N^2)$ per signal, and the paper notes that computing the eigendecomposition itself "might be prohibitively expensive for large graphs" (it is $O(N^3)$ for a dense matrix). A filter that sets an independent factor for each of the $N$ frequencies also mixes every node with every other node.

The way out, from Hammond, Vandergheynst and Gribonval (2011) and used for a graph CNN by Defferrard, Bresson and Vandergheynst (2016, called ChebNet), restricts $g_\theta$ to a polynomial of degree $K$ in the eigenvalues, written in Chebyshev polynomials $T_k$ (defined by $T_0(x)=1$, $T_1(x)=x$, $T_k(x)=2xT_{k-1}(x)-T_{k-2}(x)$). The eigenvalues are first rescaled into $[-1,1]$, where every Chebyshev polynomial stays between −1 and 1, using the largest eigenvalue $\lambda_{\max}$:

Two properties follow. Because $(U\Lambda U^\top)^k = U\Lambda^k U^\top$, a polynomial applied to the eigenvalues equals the same polynomial applied to the matrix, so (5) never needs $U$: it is computed with $K$ multiplications by the sparse matrix $\tilde{L}$, at a cost of $O(|\mathcal{E}|)$, linear in the number of edges. And entry $(i,j)$ of $L^k$ is zero when nodes $i$ and $j$ are more than $k$ edges apart, so the filtered value at a node depends only on nodes within $K$ hops. The paper calls this property K-localized. The degree $K$ sets the filter's radius the way kernel size does on an image: a 3×3 kernel reaches one pixel out, and a degree-1 graph filter reaches one hop out.³

The renormalization trick: K = 1 and a self-loop

In ChebNet $K$ is a hyperparameter (the paper's own comparison tries $K=2$ and $K=3$). Kipf and Welling ask whether a layer needs more than $K=1$. A degree-1 filter reaches one hop, and stacking two such layers reaches two hops, so depth can stand in for polynomial degree. The paper gives two reasons to prefer that. A filter with fewer parameters per layer should overfit less to local neighborhood structure on graphs whose node degrees vary widely, as citation networks and social networks do. And for a fixed compute budget, cheaper layers allow more of them.

With $K=1$ and the further approximation $\lambda_{\max}\approx 2$ (the largest possible value for the normalized Laplacian; the paper expects the trained weights to absorb the error in scale), equation (5) has two terms:

The minus sign comes from $L - I_N = -D^{-1/2}AD^{-1/2}$. The paper then ties the two parameters together, setting $\theta=\theta'_0=-\theta'_1$, to reduce overfitting and the number of operations per layer. Substituting turns both terms positive:

The operator in (7) keeps the node's own value with weight 1 and adds its neighbors' values with weights $1/\sqrt{d_i d_j}$. Its eigenvalues lie in $[0,2]$, and the paper names the consequence: "Repeated application of this operator can therefore lead to numerical instabilities and exploding/vanishing gradients." The top eigenvalue is exactly 2 on any connected graph, so a component of the signal along the top eigenvector doubles at every layer, growing $2^4=16$ times over four layers, while components with eigenvalues near 0 shrink toward zero. The paper replaces the operator with one substitution, the renormalization trick:

The left side normalizes the adjacency by the original degrees and then adds the identity. The right side adds the self-loops first and normalizes afterward, by degrees that now include them. Every entry changes: the diagonal goes from 1 to $1/\tilde{d}_i$, and an edge weight goes from $1/\sqrt{d_id_j}$ to $1/\sqrt{\tilde{d}_i\tilde{d}_j}$. The top eigenvalue drops to exactly 1, for this reason: $\hat{A}$ has the same eigenvalues as $\tilde{D}^{-1}\tilde{A}$ (the two are related by the change of basis $\tilde{D}^{1/2}$), and that matrix has nonnegative rows summing to 1, so no eigenvalue exceeds 1 in magnitude and the all-ones vector has eigenvalue exactly 1. With self-loops the bottom of the spectrum stays above −1, so the whole spectrum lies in $(-1,1]$ and stacked layers no longer amplify the top mode.

Figure 3 shows the matrices with numbers in them, on a five-node graph: a triangle 0-1-2 with a tail 0-3-4. Switch between Eq (7) and renormalized and compare the cell for the edge 0-1. Under (7) it is $1/\sqrt{3\cdot2}=0.41$; renormalized it is $1/\sqrt{4\cdot3}=0.29$, and the diagonal entries drop from 1 to between 0.25 and 0.5. The eigenvalue range moves from $[0.14, 2]$ to $[-0.21, 1]$. The row-sum column shows the renormalized rows summing to between 0.91 and 1.12. The row average button shows $\tilde{D}^{-1}\tilde{A}$ for comparison: its rows sum to exactly 1, it has the same eigenvalues as the renormalized matrix, and it is not symmetric.

A star graph isolates the effect of degree. With a hub of degree $d$ joined to $d$ leaves, the unnormalized $A+I$ has top eigenvalue $1+\sqrt{d}$, which grows without bound as the hub gets bigger. The Eq (7) operator stays at 2 whatever the degree, and the renormalized operator stays at 1. In Figure 4, drag the hub degree to see the raw curve climb, and drag the layer count to see what each top eigenvalue does to a signal after that many layers: at $d=9$ and four layers the raw operator multiplies the top mode by $4^4=256$, the Eq (7) operator by 16, and the renormalized one by 1.

Kipf and Welling justify the trick by the instability alone, and show in their ablation that it also gives the best accuracy. A spectral analysis of why the self-loop helps came two years later: Wu et al. (2019, SGC) proved that adding self-loops shrinks the largest eigenvalue of the corresponding normalized Laplacian (from about 2 to about 1.5 on Cora), which makes the filter act as a low-pass filter.⁴

The last step generalizes from one scalar signal to $C$ input channels and $F$ output channels. The single parameter $\theta$ becomes a matrix $\Theta\in\mathbb{R}^{C\times F}$:

$Z\in\mathbb{R}^{N\times F}$ is the filtered output, and the cost is $O(|\mathcal{E}|FC)$ because $\tilde{A}$ is sparse. Equation (8) followed by a nonlinearity is the layer rule (2). Its $1/\sqrt{\tilde{d}_i\tilde{d}_j}$ weights are a first-order Chebyshev filter with $\lambda_{\max}\approx2$, tied parameters, and the self-loop moved inside the normalization.

The two-layer model and its training loss

The model the paper trains stacks two layers. $\hat{A}=\tilde{D}^{-1/2}\tilde{A}\tilde{D}^{-1/2}$ is computed once, before training, and then

$W^{(0)}\in\mathbb{R}^{C\times H}$ maps the input features to $H$ hidden units, $W^{(1)}\in\mathbb{R}^{H\times F}$ maps those to $F$ class scores, and the softmax runs over each row. On Cora that is $X$ of shape $2708\times1433$, $W^{(0)}$ of shape $1433\times16$, a hidden layer of shape $2708\times16$, $W^{(1)}$ of shape $16\times7$, and an output $Z$ of shape $2708\times7$: one probability distribution over the 7 subjects for every paper, labeled or not. The loss is cross-entropy over the labeled nodes only:

$\mathcal{Y}_L$ is the set of labeled node indices, 140 of them on Cora, and $Y_{lf}$ is 1 for the true class of node $l$ and 0 otherwise. The 140 terms reach the other 2,568 nodes through $\hat{A}$, in two steps. The output row $Z_l$ of a labeled node is computed by the second $\hat{A}$ from the hidden rows of $l$ and its neighbors. Each of those hidden rows was computed by the first $\hat{A}$ from the input features of that node and its own neighbors. So $Z_l$ depends on the features of every node within two hops of $l$, and the gradient of the term for $l$ with respect to $W^{(0)}$ and $W^{(1)}$ is a sum of contributions from all of those nodes, nearly all of them unlabeled. The labels supervise the two shared weight matrices, and the same matrices then compute $Z$ for the unlabeled nodes.

Figure 5 shows the two-hop reach on a small graph, Zachary's karate club (34 members, 78 friendships), with one labeled node in each of its two factions: node 0 and node 33. These two nodes have 16 and 17 neighbors. At one layer their outputs already depend on 31 of the 34 nodes; at two layers, on all 34. Tap any other node to see its own receptive field, which on this graph reaches everything within five layers, the graph's diameter. Brightness shows the size of each node's entry in the focus node's row of $\hat{A}^k$ (the two labeled nodes' rows added together, by default).

The forward pass and one training step look like this. $\hat{A}X$ is a sparse-times-dense product, so the whole forward pass costs $O(|\mathcal{E}|CHF)$, linear in the number of edges. Training is full-batch: the entire graph is one example, and every iteration is a gradient step on all of it.

# two-layer GCN on Cora: N=2708 nodes, C=1433 features, H=16 hidden, F=7 classes
A_t   = A + I                              # add a self-loop to every node
d_t   = A_t.sum(axis=1)                    # degrees counting the self-loop
A_hat = diag(d_t**-0.5) @ A_t @ diag(d_t**-0.5)   # sparse, computed once
X     = X / X.sum(axis=1, keepdims=True)   # row-normalize the bag-of-words rows

H = relu(A_hat @ dropout(X) @ W0)          # [2708,1433] @ [1433,16] -> [2708,16]
Z = softmax(A_hat @ dropout(H) @ W1)       # [2708,16]   @ [16,7]    -> [2708,7]

loss = cross_entropy(Z[idx_train], Y[idx_train])   # 140 labeled rows only
loss = loss + 5e-4 * 0.5 * (W0**2).sum()           # L2 on the first layer only
loss.backward(); adam.step()               # gradients reach W0 and W1

The settings, from Section 5.3 of the paper: Adam with learning rate 0.01, at most 200 epochs, early stopping with a window of 10 epochs on the validation loss, dropout rate 0.5, L2 regularization $5\cdot10^{-4}$ on the first layer, 16 hidden units, and Glorot initialization with row-normalized input features. The hyperparameters were tuned on Cora and reused unchanged for Citeseer and Pubmed. The authors' TensorFlow code (github.com/tkipf/gcn) matches these, with two details the paper leaves implicit, listed in the Provenance panel: the L2 term uses TensorFlow's l2_loss, which is half the squared norm (hence the 0.5 above), and training stops when the validation loss exceeds the mean of the previous 10 epochs.

Results on Cora, Citeseer, Pubmed and NELL

Table 2 of the paper reports accuracy of 81.5% on Cora, 70.3% on Citeseer, 79.0% on Pubmed and 66.0% on NELL, the highest of every method compared on all four. The margins over the best baseline differ a lot between datasets:

• Cora: +5.8 points over Planetoid (75.7%); ICA scored 75.1%.
• NELL: +4.1 points over Planetoid (61.9%).
• Pubmed: +1.8 points over Planetoid (77.2%).
• Citeseer: +1.2 points over ICA (69.1%).

Wall-clock time to convergence is in the same table. Against Planetoid, GCN took 4 s instead of 13 s on Cora, 7 s instead of 26 s on Citeseer and 48 s instead of 185 s on NELL, roughly three to four times faster, but 38 s instead of 25 s on Pubmed. Compared with ICA, which passes predicted labels between neighbors, the paper attributes part of the gain to passing feature information in every layer.

Table 3 isolates the propagation operator. Each row trains the same model with a different matrix in place of $\hat{A}$, on the same splits. The row that drops the graph altogether, a multi-layer perceptron computing $X\Theta$, scores 55.1% on Cora, against 81.5% for the renormalized GCN: 26.4 points of accuracy come from the graph. Among the operators that do use the graph the spread is small, from 79.2% to 81.5% on Cora, and the renormalized operator is the best on all three citation datasets. It beats Chebyshev filters with $K=2$ (81.2%) and $K=3$ (79.5%), which have more parameters per layer, and the first-order variants: the two-parameter model of (6) (80.0%), the single-parameter model of (7) (79.2%) and the neighbor term alone (80.5%). Figure 6 plots the table; switch datasets with the buttons.

All of these numbers come from one fixed split per dataset, the one Planetoid published. The paper also trains on 10 random splits of the same size and reports the mean and standard error: Cora 80.1 ± 0.5, Citeseer 67.9 ± 0.5, Pubmed 78.9 ± 0.7, NELL 58.4 ± 1.7. Cora drops 1.4 points from the fixed split and NELL 7.6. Shchur et al. (2018) later showed that the ranking of graph methods can change between random splits, so differences of one or two points on the fixed split, such as the Citeseer and Pubmed margins, are weak evidence on their own.⁷

Why GCNs stay shallow

Appendix B of the paper trains models of up to 10 layers (5-fold cross-validation on Cora, Citeseer and Pubmed using all labels, 16 units per layer, 400 epochs without early stopping). Two or three layers give the best results. Beyond seven layers, training without residual connections "can become difficult". The paper gives two reasons there: each extra layer widens the neighborhood a node draws on by one hop, and the parameter count grows with depth, which invites overfitting. Section 2.2 adds a third, which applies to the un-renormalized operator: repeated application of a matrix with eigenvalues up to 2 is numerically unstable.

Later work added a fourth explanation, now the usual one, called oversmoothing (Li, Han and Wu, 2018, and after). The word does not appear in the 2016 paper. The argument: every layer replaces a node's features with a weighted sum over its neighborhood, so after many layers the features of all nodes in a connected graph converge to the same pattern, and a classifier can no longer separate them. Figure 7 runs this on the 11-node graph with no weights and no nonlinearity, only repeated multiplication by $\hat{A}$. The input is +1 on one group and −1 on the other, and the curve tracks the gap between the two groups' mean values. It falls below half its starting value at 12 layers and to about 2% at 64.

At 64 layers every node has the same sign. The signal is heading toward the top eigenvector of $\hat{A}$, whose entries are proportional to $\sqrt{\tilde{d}_i}$ and whose eigenvalue is 1: every other component is multiplied by an eigenvalue of magnitude below 1 at each layer and decays, while this one is kept. On this graph the second eigenvalue is about 0.94, so that component shrinks only by $0.94^{64} \approx 0.02$ over 64 layers and still sets the shape at that depth: the right group's hub (degree 5) holds 5.2 times the value of a degree-2 node on the left, where the top eigenvector would give $\sqrt{6/3} \approx 1.41$. The values settle onto $\sqrt{\tilde{d}_i}$ near 150 layers. (With the row average $\tilde{D}^{-1}\tilde{A}$ the limit would be a constant vector instead.) A trained network has weight matrices and ReLUs between the multiplications, so the decay in a real model is not this clean. The paper's remedy for depth is a residual connection, borrowed from ResNet:

Adding the layer's input back to its output lets each layer carry the previous representation forward, and in Appendix B the residual models stay trainable out to the 10 layers tested.

The Weisfeiler-Lehman view

Appendix A reads the layer rule a second way, with no spectral theory. The 1-dimensional Weisfeiler-Lehman algorithm is a classic test for whether two graphs could be the same graph with the nodes renamed. It gives every node a color, then repeatedly replaces each node's color with a hash of its own color and the multiset of its neighbors' colors. After a few rounds, nodes in structurally different positions usually have different colors. Replace the hash with a normalized sum, a weight matrix and a nonlinearity, and the update becomes

In (12), $\mathcal{N}_i$ is the set of node $i$'s neighbors and $d_i$ its size; with a self-loop counted as a neighbor this is equation (2) written for one node. The paper calls the GCN, "loosely speaking", a differentiable and parameterized generalization of the Weisfeiler-Lehman algorithm. It is weaker as a test: a weighted sum can give the same result for two different multisets of neighbor colors, where a hash would not. Xu et al. (2019, the GIN paper) proved that message-passing networks of this kind are at most as discriminative as the Weisfeiler-Lehman test, and that GCN cannot distinguish some simple structures that the test can.

The appendix uses this view to predict that a GCN with random, untrained weights already produces useful node embeddings, because the Weisfeiler-Lehman-style update separates nodes by their position in the graph. It tests the prediction on the karate club with a three-layer GCN and no node features ($X=I_N$, so each node starts as its own one-hot vector):

The weight matrices are drawn at random with Glorot initialization and never trained. The random-weight embeddings are, in the paper's words, comparable to those of DeepWalk, which uses "a more expensive unsupervised training procedure".

Appendix A.2 then trains that model with one labeled node per class and watches the two-dimensional output move. Figure 8 reproduces the experiment in a smaller form, computed live in your browser: a featureless two-layer GCN with one labeled node in each of the club's two factions.⁶ The output starts as a cluster near the origin; press play and the two factions separate, using only the two labels and the edges.

Limitations the paper lists

Section 7.2 names three. Memory: full-batch training holds the whole graph and its activations, so memory grows linearly with the dataset. Mini-batch training is possible, but an exact mini-batch for a $K$-layer GCN must include every node within $K$ hops of the batch, and the paper notes that very large, densely connected graphs may need further approximations. Directed edges and edge features: the model handles undirected graphs only, weighted or unweighted. For NELL the authors convert the directed, relation-labeled graph into an undirected bipartite graph with an extra node for each original edge.

Fixed self-weight: the approximations of Section 2 assume locality and give a node's own features the same weight as each neighbor's. The paper suggests a trade-off parameter in the self-loop,

learned by gradient descent, which would play the role that $\lambda$ plays in the regularized loss (1). It is proposed as future work. Inductive use, on graphs or nodes unseen in training, is outside its scope altogether.