States, Actions, and Rewards

The three raw ingredients of any reinforcement learning problem: what the agent sees, what it can do, and how it is scored.

The three signals

Every reinforcement learning problem is built from three signals exchanged between agent and environment: states, actions, and rewards. Choosing how to represent each one is often more consequential than the choice of algorithm.

States

A state describes the situation the agent faces. It may be discrete (a board position) or continuous (joint angles, temperatures, magnetic-field readings). A good state representation is compact, informative, and as close to Markov as possible. Poor state design either omits relevant variables or drowns the learner in irrelevant detail.

Actions

Actions are the agent's levers on the world. Discrete action spaces enumerate choices (move left, fire, wait). Continuous action spaces specify real-valued controls (a torque, a voltage, a coil current). Some algorithms suit discrete actions (Q-learning, DQN) while others target continuous control (DDPG, SAC, PPO).

Rewards

The reward is a scalar that scores each transition. It is the sole definition of the goal: the agent optimizes reward, not the designer's unstated intent. Rewards can be dense (feedback every step) or sparse (nonzero only at rare successes). Sparse rewards are honest but hard to learn from; dense rewards learn faster but risk encoding the wrong objective.

A design example

For a simulated control task keeping a quantity near a target, the state might include the current value and its rate of change, the action might be a control adjustment, and the reward might penalize deviation from target and large control effort. Even a task this simple forces choices about scaling, clipping, and what counts as failure — see reward shaping.