Deep Q-Network (DQN)

A Deep Q-Network (DQN) is a way for an agent to learn which action is worth taking from rich inputs such as game screens. Rather than storing a separate score for every possible situation—which becomes impossible when there are millions of images or states—it uses a neural network to estimate those scores.

What the network learns
DQN learns an action-value function, written Q(s, a): the expected total future reward from taking action a in state s and then behaving well afterward. Given an Atari screen, for example, the network outputs one Q-value per available joystick action. The agent usually selects the highest-valued action, but deliberately takes random actions part of the time—epsilon-greedy exploration—to discover alternatives it has not tried.

How DQN trains without labels
After each interaction, DQN stores a transition: state, action, reward, next state, and whether the episode ended. It trains its Q-value toward a target based on the observed reward plus its estimate of the best next action:

Immediate reward says what happened now.
Bootstrapping adds the discounted value predicted for the next state.
Experience replay samples old transitions randomly from a buffer, breaking up highly correlated consecutive game frames.
A separate, slowly updated target network supplies the training target, preventing the network from chasing a target that changes every gradient step.

Why these details matter
Plain neural-network Q-learning is unstable because the same network both predicts values and defines the values it is trying to match; errors can amplify rather than fade. Replay and target networks were the practical breakthrough behind DQN’s strong Atari results. Yet DQN remains best suited to discrete actions: “left,” “right,” or “fire,” not a robot’s continuous motor torques. It can also overestimate action values, learn reward-function loopholes, or fail when its training dynamics shift. Variants such as Double DQN reduce overestimation, while modern libraries such as Stable Baselines3 provide DQN implementations for Gymnasium-style environments.

Deep Q-Network (DQN) is a value-based reinforcement-learning algorithm that uses a neural network to approximate the action-value function, Q(s,a): the expected discounted return from taking an action in a state. It learns from stored experience and uses a separate target network to stabilize Q-learning updates. DQN enables learning action policies directly from high-dimensional inputs, such as game pixels, where tabular Q-learning is infeasible.

Imagine learning an old video game by playing it again and again. You try different buttons, notice which moves earn points or keep you alive, and gradually learn what tends to be a good choice in each situation.

A Deep Q-Network (DQN) is an AI version of that idea. It learns from trial and error which action is likely to lead to the best future reward: turn left or right, jump or wait, accelerate or brake. “Deep” means it uses a large pattern-finding system that can learn directly from complicated input, such as game-screen pixels. DQN mattered because it showed that an AI could learn to play many games from experience rather than being given the rules.