LONG-SHORT TERM MEMORIES (LSTMs)
The figure above is diagram of how LSTMs operate. Long-Short Term Memories (LSTMs) are a type of Recurrent Neural Network (RNN) capable of learning and remembering long-term dependencies. The default behavior is to recall past information for extended periods of time. LSTMs keep track of information over time. Because they remember previous inputs, they are useful in time-series prediction. LSTMs have a chain-like structure with four interacting layers that communicate in a distinct manner. LSTMs are commonly used for speech recognition, music composition, and pharmaceutical development, in addition to time-series predictions. There is three steps how LSTMs work: First, they forget irrelevant parts of the previous state. Next, they selectively update the cell-state values. Finally, the output of certain parts of the cell state.
LSTMs has been reviewed at this sections LSTMs Neural Networks. We can learn more explanations by applying different code in Deep Learning part.
Components of LSTMs
So the LSTM cell contains the following components:
# Forget Gate “f” ( a neural network with sigmoid ).
# Candidate layer “C" ( a NN with Tanh ).
# Input Gate “I” ( a NN with sigmoid ).
# Output Gate “O” ( a NN with sigmoid).
# Hidden state “H” ( a vector ).
# Memory state “C” ( a vector).
# Inputs to the LSTM cell at any step are Xt (current input) , Ht-1 (previous hidden state ) and Ct-1 (previous memory state).
#Outputs from the LSTM cell are Ht (current hidden state ) and Ct (current memory state).
Working of gates in LSTMs
First, LSTM cell takes the previous memory state Ct-1 and does element wise multiplication with forget gate (f) to decide if present memory state Ct. If forget gate value is 0 then previous memory state is completely forgotten else f forget gate value is 1 then previous memory state is completely passed to the cell ( Remember f gate gives values between 0 and 1 ).
Ct = Ct-1 * ft
Calculating the new memory state:
Ct = Ct + (It * Ct)
Now, we calculate the output:
Ht = tanh(Ct)
Full Introduction to Recurrent Neural Networks LSTM (Stock Market Predictions) using Gated Recurrent Units in Python with Keras
Before you start, you can download numpy using this version (see figure above).
This tutorial will use LSTMs for predicting the price of stocks of IBM for the year 2017. You can download csv file from here. Then, see the full code below, the code of CNN for Deep Learning using Fashion MNIST: