The planning horizon dimension is how far ahead in time the agent plans. For example, consider a dog as an agent. When a dog is called to come, it should turn around to start running in order to get a reward in the future. It does not act only to get an immediate reward. Plausibly, a dog does not act for goals arbitrarily far in the future (e.g., in a few months), whereas people do (e.g., working hard now to get a holiday next year).

How far the agent “looks into the future” when deciding what to do is called the planning horizon. For completeness, we include the non-planning case where the agent is not reasoning in time. The time points considered by an agent when planning are called stages.

In the planning horizon dimension, an agent is one of the following:

• •

  A non-planning agent is an agent that does not consider the future when it decides what to do or when time is not involved.

• •

  A finite horizon planner is an agent that looks for a fixed finite number of stages. For example, a doctor may have to treat a patient but may have time for a test and so there may be two stages to plan for: a testing stage and a treatment stage. In the degenerate case where an agent only looks one time step ahead, it is said to be greedy or myopic.

• •

  An indefinite horizon planner is an agent that looks ahead some finite, but not predetermined, number of stages. For example, an agent that must get to some location may not know a priori how many steps it will take to get there, but, when planning, it does not consider what it will do after it gets to the location.

• •

  An infinite horizon planner is an agent that plans on going on forever. This is often called a process. For example, the stabilization module of a legged robot should go on forever; it cannot stop when it has achieved stability, because the robot has to keep from falling over.