Cell State

Think of an LSTM cell as carrying a running notebook through a sequence. Its cell state is the notebook: a protected pathway for information that should survive across many steps, rather than being rewritten from scratch each time a new input arrives.

The long-term memory pathway

At time step t, the cell state, written c_t, is updated by two learned gates. The forget gate chooses how much of the previous state c_t−1 to retain, while the input gate chooses how much newly proposed content to write. In compact form:

c_t = f_t * c_(t-1) + i_t * g_t

Here, f_t and i_t contain values between 0 and 1, and g_t is candidate memory content. Element-by-element multiplication lets the LSTM preserve one piece of information while replacing another.

Cell state versus hidden state

The hidden state is the LSTM’s exposed, short-term working output: it is used to produce predictions and help compute the next update. The cell state is more private and persistent. An output gate decides which parts of the cell state become visible in the hidden state. This separation gives the model a place to store a long-running dependency without forcing every stored detail into every output.

Why it makes training work

A vanilla RNN repeatedly transforms its state, so gradients can shrink or explode as they are backpropagated through many time steps. The cell state has an additive update path, allowing gradients to travel backward with less distortion when the forget gate stays near 1. For example, an LSTM can retain an early condition until a later step needs it, then forget it once it is irrelevant. Poor gate biases or unstable optimization can still cause memory to be erased too eagerly or retained uselessly, but the cell state is the mechanism that makes controlled long-range memory practical.