Training a Small CNN on Toy Images

Build and train a compact convolutional network to separate two synthetic image classes, and read the convergence from the loss curve.

Problem

We generate small grayscale images: class 0 has a vertical bar, class 1 has a horizontal bar, both with noise. A convolutional network should learn edge-oriented filters that distinguish the two, which is the smallest honest demonstration of what convolutions buy over fully connected layers.

Architecture

One convolution layer with a few 3x3 filters, a ReLU nonlinearity, global average pooling, then a linear classifier. Parameter sharing across spatial positions means the same filter detects a bar wherever it appears, which a dense layer cannot do without seeing every position in training.

import numpy as np
rng=np.random.default_rng(0)
def make(k,n=200):
    X=rng.normal(0,0.1,(n,8,8)); y=np.zeros(n,int)
    for i in range(n):
        if i%2: X[i,3,:]+=1.0; y[i]=1     # horizontal bar
        else:   X[i,:,3]+=1.0; y[i]=0     # vertical bar
    return X,y
Xtr,ytr=make(0); 
# a real run uses torch; here the point is the recipe:
# conv3x3 -> relu -> global-avg-pool -> linear -> softmax, trained by SGD on cross-entropy
print(Xtr.shape, ytr[:6])

Training signal

With cross-entropy loss and stochastic gradient descent the loss falls from about ln(2)=0.69 (random guessing on two classes) toward near zero within a few dozen epochs. Learned filters resemble oriented edge detectors: one responds strongly to vertical structure, another to horizontal.

• Convolution plus pooling gives translation tolerance for free, the reason CNNs generalize from few examples on structured images.
• Watch the gap between training and validation loss; if validation stalls while training keeps dropping, the model is memorizing noise.
• Kronos uses similar convolutional encoders on camera and bolometer image data to flag off-normal plasma shapes.