This makes sense! Convolutions are translationally invariant because the filters slide over the image horizontally and vertically. But they are not rotationally invariant because the filters don’t rotate. The network, thus, seems to need several similar filters in different orientations to detect objects and patterns that are differently oriented.

The code

The idea is the following: we start with a picture containing random pixels. We apply the network in evaluation mode to this random image, calculate the average activation of a certain feature map in a certain layer from which we then compute the gradients with respect to the input image pixel values. Knowing the gradients for the pixel values we then proceed to update the pixel values in a way that maximizes the average activation of the chosen feature map.

I know that this might sound confusing so let’s explain it again in different words: The network weights are fixed, the network will not be trained, and we try to find an image that maximizes the average activation of a certain feature map by performing gradient descent optimization on the pixel values.

This technique is also used for neural style transfer.

In order to implement this we will need:

A random image to start with A pre-trained network in evaluation mode A nice way to access the resulting activations of any hidden layer we are interested in A loss function to compute the gradients and an optimizer to update the pixel values

Let’s start with generating a noisy image as input. We can do this i.e. the following way: img = np.uint8(np.random.uniform(150, 180, (sz, sz, 3)))/255 where sz is the height and width of the image, 3 is the number of color channels, and we divide by 255 because it is the maximum value a variable of type uint8 can store. Play with the numbers 150 and 180 if you want more or less noise. We then convert this to a PyTorch variable that requires gradients using img_var = V(img[None], requires_grad=True) (this is fastai syntax). The pixel values require gradients as we want to optimize them using backpropagation.

Next, we need a pre-trained network in evaluation mode (which means that the weights are fixed). This can be done with model = vgg16(pre=True).eval() and set_trainable(model, False) .

Now, we need a way to access the features from one of the hidden layers. We could truncate the network after the hidden layer we are interested in so that it would become the output layer. There is, however, a nicer way to solve this problem in PyTorch called a hook which can be registered on a PyTorch Module or a Tensor . To understand this, you have to know:

A Pytorch Module is the base class for all neural network modules. Every layer in our neural network is a Module . Every Module has a method called forward that calculates the output of the Module for a given input.

When we apply our network to our noisy image the forward method of the first layer takes the image as input and calculates its output. This output is the input to the forward method of the second layer and so on. When you register a forward hook on a certain layer the hook is executed when the forward method of that layer is called. Ok, I know this sounds confusing. What I want you to take from this is the following: when you apply your network to an input image the first layer calculates its output, then the second, and so on. When we reach a layer for which we registered a hook, it not only calculates its output but also executes the hook.

So what is this good for? Let’s say we are interested in the feature maps of layer i. We register a forward hook on layer i that, once the forward method of layer i is called, saves the features of layer i in a variable.

The following class does this:

When the hook is executed, it calls the method hook_fn (see constructor). The method hook_fn saves the layers output in self.features . Note, that this tensor requires gradients because we want to perform backpropagation on the pixel values.

How would you use a SaveFeatures object?

Register your hook for layer i with activations = SaveFeatures(list(self.model.children())[i]) and after you applied your model to the image with model(img_var) you can access the features the hook saved for us in activations.features . Remember to call the method close to free up used memory.

Great, we can now access the feature maps of layer i! The feature maps could i.e. have the shape [1, 512, 7, 7] where 1 is the batch dimension, 512 the number of filters/feature maps and 7 the height and width of the feature maps. The goal is to maximize the average activation of a chosen feature map j. We, therefore, define the following loss function: loss = -activations.features[0, j].mean() and an optimizer optimizer = torch.optim.Adam([img_var], lr=lr, weight_decay=1e-6) that optimizes the pixel values. Optimizers by default minimize losses so instead of telling the optimizer to maximize the loss we simply multiply the mean activation by -1. Reset the gradients with optimizer.zero_grad() , calculate the gradients of the pixel values with loss.backward() , and change the pixel values with optimizer.step() .

We now have everything that we need: we started with a random image, defined a pre-trained network in evaluation mode, registered a forward hook to access the features of layer i, and defined an optimizer and a loss function that allows us to change the pixel values in a way that maximizes the mean activation of feature map j in layer i.

Great, let’s look at an example of what this gives us:

layer 40, filter 265

Wait, this is not what we wanted right? This was supposed to result in the chain pattern I showed you before. If you pinch your eyes you might be able to guess where “the chains” might be. However, we must have ended up in a very poor local minimum and have to find a way to guide our optimizer towards a better minimum/better-looking pattern. In contrast to the generated patterns I showed you before, this picture is dominated by a high-frequency pattern that resembles adversarial examples. So, what could we do to fix this? I experimented with different optimizers, learning rates, and regularizations but nothing appeared to reduce the high-frequency patterns.

Next, I varied the size of the noisy input image.