Jerry the mouse repeatedly outsmarts hapless Tom the cat in the classic cartoon Tom & Jerry. Now, researchers from Two Six Labs and Stanford Schnitzer Lab have developed a deep learning system designed to explore the workings of the mouse mind and predict behavior by processing brain-based electrical activity with a neural network.

Researchers focused on the mouse’s hippocampus, a part of the brain involved in learning, memory and navigation that contains neurons known as “place cells” which fire in response to the mouse’s location within its environment. A tiny microscope was attached to the mouse’s head to record the fluorescence of a brain-borne dye that tracks neurons when they fire. Researchers produced a dataset capturing a mouse’s neural activity as it moved around in a 45×60 cm box containing visual landmarks.

Model predictions (blue dot) and the labeled position of the mouse (red dot).

Researchers then trained a neural network to predict mouse location based on recent neuronal firing patterns. They used the first 80 percent of their observations as training data. Given only the neuronal activity, the neural network was tasked with predicting the mouse’s location for the remaining 20 percent of observations. In the experiments, a simple dense neural network with a regression output layer performed well, returning an average prediction error of only 4 cm.

Researchers sectioned the box into a 1 cm grid and designed a deep neural net model with convolutional layers to measure the certainty of their predictions based on a classification task that predicts the grid square the mouse occupies in the box. A heat map then shows prediction strength.

The blue cloud represents the areas of the box with the highest predicted probability that the mouse is there. The red dot is the mouse’s labeled location.

Researchers also developed a new loss function to train their model to understand that a nearly-correct prediction is better than a far-off prediction. This model performed similarly to the point prediction model, with an average error of 5 cm, but includes much more information about alternative predictions and the model’s certainty.

This concept of uncertainty was also applied to predict future activity in increments of 1, 2, or 3 seconds. The hippocampal neurons that researchers modeled enabled prediction of the mouse’s current position with 75 percent balanced accuracy, and predicted its position in 2 seconds with 66 percent accuracy.

The researchers plan to produce more complex behavioral datasets to apply these methods for example in predicting a mouse’s actions in a maze or its reactions to certain themed images. By studying mouse behavior in this way, researchers believe they might also learn more about human beings and contribute to a better understanding of biological neural networks.

Click here for more information on the research.