An agent living in an environment usually does not have the luxury of offline thinking while the world waits for it to consider the best option. However, offline reasoning, where the agent can reason about the best thing to do before having to act, is often a simplifying assumption.

The interaction dimension considers whether the agent does

• •

  offline reasoning where the agent determines what to do before interacting with the environment, or

• •

  online reasoning where the agent must determine what action to do while interacting in the environment, and needs to make timely decisions.

Some of the algorithms are simplified to do what could be called pure thought, without interaction with the environment. More sophisticated agents reason while acting; this includes long-range strategic reasoning as well as reasoning for reacting in a timely manner to the environment.