Reinforcement Learning

Reinforcement learning (RL) is the paradigm in which an agent learns by doing. Placed in an environment, the agent observes its state, takes an action and receives a numerical reward in return; by repeating that loop through trial and error, it shapes its behaviour to maximise the cumulative reward over time rather than any single immediate payoff.

The contrast with supervised learning is fundamental. There, a model learns from labelled examples that spell out the correct answer; here, no one reveals which action was best. The agent has only a reward signal —often sparse and delayed— and must discover for itself which sequence of decisions leads to the best outcome. It learns from the consequences of its actions, not from a teacher.

The building blocks and the Markov decision process

RL is formalised as a Markov decision process (MDP): a set of states, a set of actions, a dynamic that governs how one state leads to the next, and a reward function. The Markov property requires that the future depend only on the present state, not on the entire past history.

Two central pieces operate within that frame. The policy is the agent's strategy: the rule that, given a state, decides which action to take. The value function estimates how much reward can be expected in the long run from a state (or from a state-action pair), acting as a compass for improving the policy. The goal of learning is to find a policy that maximises expected value. Richard Sutton and Andrew Barto set out this vocabulary in the field's standard text, «Reinforcement Learning: An Introduction».

Explore or exploit: the dilemma and its methods

Because the agent learns while it acts, it faces a constant tension: exploration versus exploitation. To exploit is to pick the action that looks best right now; to explore is to try alternatives whose value is still unknown, in case they turn out better. Too much exploitation traps the agent in a mediocre optimum; too much exploration squanders reward. Balancing the two is one of the field's founding problems.

Three families stand out among the classical methods. Temporal-difference (TD) learning updates its estimates from other estimates, without waiting for the episode to end. Q-learning, formulated by Christopher Watkins, learns the value of each action directly and converges to the optimal policy even when the agent explores sub-optimally. And policy-gradient methods adjust the policy's parameters directly in the direction that increases expected reward, bypassing an explicit value function.

From the game board to RLHF

For years RL shone in closed domains —board games, video games, robot control— where the reward is clear and games can be played by the million. When it is paired with deep neural networks to handle high-dimensional environments, we speak of deep reinforcement learning, which this encyclopedia covers in its own entry.

Its relevance in 2026 reaches well beyond play. RL is the foundation of reinforcement learning from human feedback (RLHF): people compare a large language model's responses, those preferences train a reward model, and RL then tunes the model to produce more helpful, better-aligned answers. The idea, introduced by Paul Christiano and colleagues in 2017, is now a standard part of building large models.

Open fronts remain. Reward design is still delicate: a poorly framed signal pushes the agent to exploit shortcuts instead of solving the task. And sample efficiency —how many interactions an agent needs in order to learn— limits its use in the physical world, where every trial costs time and resources.