What is a neural network?

To understand what a neural network is, we need to first understand the basic concept of machine learning. In supervised learning, we are almost always trying to make a prediction, based on a series of data we have.

For example, we have a basket of cherries and blueberries. At position [X1_1, X2_1], we have a cherry; at position [X1_2, X2_2], we have a blueberry; as so on. The machine learning problem we are trying to solve is, if we pick up the fruit at position [X1_n, X2_n], will this be a cherry, or a blueberry?

Imagine if the cherries and blueberries are arrange as the following, we can simply draw a straight line to separate them. But in real world problems, the distributions are usually much more complex.

The best place to illustrate this is Google’s Neural Network Playground. In here, we can play with data and train our neural network in real time! Let’s head to the Playground and continue on our cherry-blueberry problem.

We notice that in here, we can get very complex cherry-blueberry distributions.

For example, we can get Circle, XOR, Gaussian, and even Spiral fruit matrices!

We’ll soon notice that the first 3 set of distributions can be separated relatively easily, but for the spiral fruit matrix, there is no easy way to draw a line and separate our blueberries and cherries.

If this is a traditional ML problem, ML scientists will start banging their heads on this very tedious process called feature engineering. Aka, they will take the 2 input data we have, X1 and X2, and dress them up differently. The following shows X1, X2, X1^2, X2^2, X1*X2, sin(X1), sin(X2) respectively.

If we only use X1 and X2 as the input, every layer of our neural network, will perform a linear combination of them, and pass the result to the next layer.

And the next layer will take the result of the previous layer, and perform another linear combinatin of them, and so on.

Now that we know the concept, let’s try the traditional feature engineering approach and feed all 7 engineered features into the net. In this case we will use a very shallow net, with just 1 layer of neurons.

The final result of this training is as the following:

We can tell that we almost perfectly separated our cherries and blueberries, great!

But in this approach, we have spent a lot of time engineering the features. In this case, we hve relatively simple raw input, X1 and X2. But what if we have hundreds of features? What if we have pixel values of an image? As our features become more and more complex, eventually feature engineering will not scale. Luckily we have neural network to the rescue! Now let’s see the real magic of a neural net!

The greatest magic of a neural network is that, we no longer have to spend hours engineering the features. With a deep network, we can feed the raw features into the network, and let it learn. With enough hidden layers and neurons, the neural net wil figure it out megically!

Let’s see the magic in action. In this experiment, we will only input X1 and X2. But instead, we will configure a 6 layer, 8 neuron per layer network:

The final result of this training is as the following:

We can see that even without engineering the features, the neural net is able to learn how to separate cherries and blueberries!

In Episode 2, Scientist tries to fix TG700 using a neural network. How does this network work?

We know that when opening doors, TG700 the Atomo has data such as Speed, Direction, and Distance. He then uses these data to determine when is the best timing to open the door.

In the traditional approach, we will have to engineer the features. For example, combining Speed and Direction to get Velocity, and combining Speed and Distance to get time to door:

But our Scientist is very smart! Instead of spending hours engineering the features, he just uses a neural net and lets it learn by itself.

With the power of neural net, TG700 is becoming a lot smarter! To find out what happens to the adventure of Atomo next, stay tuned for new episodes every Monday!