Reset Gate (GRU)

A GRU processes a sequence one step at a time while carrying a compact memory called its hidden state. The reset gate is the part that decides how much of that existing memory should be ignored when the network forms a fresh candidate memory for the current step.

How the gate works
At time step t, a GRU receives the current input xₜ and previous hidden state hₜ₋₁. Its reset gate is computed as:

rₜ = sigmoid(Wr xₜ + Ur hₜ₋₁ + br)

The sigmoid makes each element of rₜ lie between 0 and 1. The gate then filters the old state before the GRU proposes a new one:

h̃ₜ = tanh(Wh xₜ + Uh (rₜ ⊙ hₜ₋₁) + bh)

Here, ⊙ means element-wise multiplication. A reset value near 0 suppresses a feature of the old memory; a value near 1 preserves it for the candidate calculation. This is selective, not an all-or-nothing erasure: different hidden-state features can be reset independently.

What it is for during training

When the current input marks a sharp context change, the reset gate lets the GRU build a candidate state without being dominated by stale history.
When prior context remains useful, it leaves that information available to the candidate computation.
Its companion, the update gate, separately decides how much of that candidate actually replaces the stored hidden state. Reset controls candidate construction; update controls state retention.

For example, after a sequence segment changes topic or regime, low reset-gate values can prevent irrelevant earlier features from contaminating the next representation. In a PyTorch GRU, these gates are learned automatically through backpropagation through time. Gating also creates easier routes for useful information and gradients than a plain RNN, reducing—but not eliminating—the risk that long-range learning stalls as gradients fade.

In a Gated Recurrent Unit (GRU), the reset gate controls how much of the previous hidden state contributes when computing the candidate new state. Values near zero suppress past information, allowing the unit to reset its context; values near one retain it. This selective forgetting lets GRUs model changing dependencies in sequences while reducing interference from irrelevant history.

Imagine reading a story while keeping only the details that still matter. When the story moves from one scene to another, you may stop thinking about the old location so you can focus on what happens next.

A reset gate is the part of a GRU—a kind of AI with a short-term memory for sequences—that makes this choice. It helps the network decide how much of its earlier memory to ignore when interpreting new information. This is useful for things like language, where a new sentence can make old details less relevant. By letting go at the right moments, the AI can pay closer attention to the current context.