I recently blogged about my inability to determine the next number in a sequence using Gradient Descent (https://agentanakinai.wordpress.com/2019/04/23/sequence/). Thanks to feedback from Reddit’s r/neuralnetworks community, it seems that the best way to solve this problem might be to use Long-Short Term Memory (LSTM).

The best source of information about LSTM that I’ve found so far is this video by Siraj Raval:

Fortunately, I think I understand the basic concept: layers of Gradient Descent. Three gates using Gradient Descent make up an LSTM cell using Gradient Descent, which is then part of a neural network using Gradient Descent. Hopefully that’s right. There’s a lot of Python code to review, and I will soon begin translating it to C to be sure that I understand everything.

I hope to blog very soon about how my revised, third neural network successfully predicted the next number in the sequence.