An extremely brief introduction.

If you can recall back to high school biology you may remember learning about nerve cells in the brain called neurons. We learned that neurons transmit nerve impulses across vast networks of other interconnected neurons! These same networks were the inspiration for the creation of artificial neural networks (ANN), the magical building blocks of deep learning.

In 1943, Warren McCulloch and Walter Pitts created a computational model (known to some as MCP) for neural networks based on threshold logic. The next major development in neural networks was the concept of a perceptron in 1958, which is an MCP neuron — where the inputs are first passed through some “preprocessors,” which are called association units.

This led to two different pathways of further research. One that focused on furthering our understanding of biological neurons and the other focused on its application to artificial intelligence.

Illustration of a biological neural network (top) and an artificial neural network (bottom)

A common misunderstanding of neural networks is that they are themselves algorithms. In actuality, neural networks are frameworks for different machine learning algorithms to work in tangent to process data inputs!

These frameworks, or systems, “learn” to perform tasks generally without any particular rules being hard coded into them. For example, an image recognition system that has been given a set of cars may be able to recognize the make of a particular input car after “learning” from examples where a car imagery + model labels are fed into the system.

As mentioned above, ANN are very similar in structure to biological neural networks. Like biological neurons, that are connected through synapses, ANN’s are connected through what are called “edges”. Each edge typically representing a weight that is adjusted as the framework learns.

Simplified artificial neural network

In the image above, each line connecting each input (x1, x2, x3…xn) are called “edges” which represent both the weights (the strength of a particular connection) between each input and what neurons its connected to in the hidden layers (h1…hm), and each hidden layer and it’s corresponding output (y1, y2, y3…). These weights increase or decrease a signal as it passes from a set of inputs to the hidden layers and finally to the outputs.

As an example, the input vectors might represent the characteristics of a customer at a bank and the output vectors might represent a prediction of whether or not that customer is likely to default on the loan they are applying for.

While you continue on your journey to understand just how ANNs work and their applications, you’ll find that they’re full of hidden complexities that are begging for further exploration! Furthermore, there are lots of different types of artificial neural networks — (CNN, RNN, DBN) each with their own advantage/disadvantage.

As the development of ANN’s continued, research and development began to stray away from what many would have considered its original goal — developing networks that solved problems the way that biological neural networks do — and attention began to move towards building very specific frameworks that focused on very specific use cases like image recognition, social network filtering, translation, game playing and medical diagnosis.

And there you have it! Are your curiosity senses tingling? We’ll be diving into more detail regarding each of the above use cases and types of neural networks over the next month and would love to have you read along!

Until next time,

🤖