Graph Neural Networks

GNNs extend deep learning to graph-structured data by learning node representations from the pattern of connections around each node.

Learning on graphs

Many datasets are naturally graphs: molecules (atoms and bonds), social networks, citation graphs, road maps, and physical meshes. These have no fixed grid or sequence order, so CNNs and RNNs do not apply directly. A graph neural network operates on nodes, edges, and their connectivity, learning a vector representation for each node that reflects both its own features and the structure of its neighborhood.

The permutation-invariance requirement

Kronos motion — data assimilation

A graph's nodes have no canonical ordering, so any operation on them must give the same result regardless of how nodes are numbered. GNN layers achieve this by aggregating neighbor information with symmetric functions such as sum, mean, or max, which ignore order. This permutation invariance is the defining constraint that distinguishes graph architectures from grid- or sequence-based ones.

Message passing

Most GNNs follow the message passing framework. In each layer, every node collects messages from its neighbors, aggregates them with a symmetric function, and updates its own representation using that aggregate and its previous state. Stacking k layers lets information travel up to k hops across the graph, so a node's final representation summarizes its k-hop neighborhood.

python

# one message-passing layer (mean aggregation)
# h_v' = update( h_v , mean_{u in N(v)} message(h_u) )

Common variants

Graph Convolutional Network (GCN): a normalized average of neighbor features.
GraphSAGE: samples a fixed number of neighbors to scale to large graphs.
Graph Attention Network (GAT): weights neighbors by learned attention.
Message Passing Neural Network (MPNN): a general framework covering many of the above.

Tasks and applications

GNNs perform node classification (labeling users or atoms), link prediction (recommending connections), and graph-level prediction (predicting a molecule's property). They power drug discovery, recommendation, traffic forecasting, and physics simulation on meshes. In engineering, GNNs act as fast surrogates for mesh-based simulations: trained on solver output, they predict fields on unstructured grids far faster than the original solver, useful for exploring designs before committing to a full computation.

Limitations

Stacking many layers can cause over-smoothing, where all node representations converge and become indistinguishable. Very large or dense graphs strain memory. Long-range dependencies are hard because information must pass hop by hop. These challenges drive ongoing work on deeper, more expressive, and more scalable graph architectures.

Operate on nodes, edges, and connectivity.
Must be invariant to node ordering.
Message passing aggregates k-hop neighborhoods.
Used for molecules, recommendation, and mesh simulation.