In this article, we’re going to introduce self-organizing maps. We assume the reader has prior experience with neural networks.

Plan of Attack:

Introduction to SOMs How SOMs work Components of Self Organization Steps of training a SOM SOM implementations Introduction to MiniSom Build a Simple SOM using minisom

Introduction to SOMs

Self-organizing maps are a class of unsupervised learning neural networks used for feature detection. They’re used to produce a low-dimension space of training samples. Therefore, they’re used for dimensionality reduction.

SOMs differ from other artificial neural networks because they apply competitive learning as opposed to error correlated learning, which involves backpropagation and gradient descent. In competitive learning, nodes compete for the right to respond to the input data subset. The training data usually has no labels and the map learns to differentiate and distinguish features based on similarities.

How SOMs work

The figure below illustrates how we train a self-organizing map. The purple blob is the distribution of the training data. The small white disc is the current training datum drawn from that distribution. At first, the SOM nodes are arbitrarily positioned in the data space. The node (highlighted in yellow) nearest the training datum is selected. It’s moved towards the training datum, as are its neighbors on the grid. After many iterations, the grid tends to approximate the data distribution (right).

The Euclidean distance to all weight vectors is computed when we feed the training data into the network. The neuron whose weight vector is most similar to the input is called the best matching unit (BMU). The weights of the BMU and neurons close to it in the SOM grid are adjusted towards the input vector. Once the BMU has been determined, the next step is to calculate which of the other nodes are within the BMU’s neighborhood.

Components of Self Organization

Initialization: all connection weights are initialized to random values.

Competition: Output nodes compete against themselves to be activated. Only one of them is activated at a time. The activated neuron is called a winner-takes-all neuron. Because of this competition, the neurons are forced to organize themselves, forming a self-organizing map (SOM).

Cooperation: The spatial location of a topological neighborhood of excited neurons is determined by the winning neuron. This provides the basis for cooperation among neighboring neurons.

Adaptation: Excited neurons decrease individual values of the discriminant function. This is done in relation to the input pattern via suitable adjustment of the associated connection weights. This way, the response of the winning neuron to the subsequent application of a similar input pattern is enhanced. The discriminant function is defined as the squared Euclidean distance between the input vector x and the weight vector wj for each neuron j:

Steps for training a Self-Organizing Map

Training a self-organizing map occurs in several steps:

1. Initialize the weights for each node. The weights are set to small standardized random values.

2. Choose a vector at random from the training set and present to the lattice.

3. Examine every node to calculate which one’s weight is most like the input vector. This will allow you to obtain the Best Matching Unit (BMU). We compute the BMU by iterating over all the nodes and calculating the Euclidean distance between each node’s weight and the current input vector. The node with a weight vector closest to the input vector is marked as the BMU.

4. Calculate the radius of the neighborhood of the BMU. Nodes found within the radius are deemed to be inside the neighborhood of the BMU.

5. Weights of the nodes found in step 4 are adjusted to make them more like the input vector. The weights of the nodes closer to the BMU are adjusted more.

6. Repeat step 2 for N iterations.

Self-Organizing Map Implementations

SOMs are commonly used in visualization. Below is a visualization of the world’s poverty data by country. The countries with higher quality of life are clustered towards the upper left while the most poverty-stricken nations are clustered towards the lower right.

Other implementations of self-organizing maps include: