Self motivated and autonomous exploration and skill acquisition

One of the striking differences between current reinforcement learning algorithms and early human learning is that animals and infants appear to explore their environments with autonomous purpose, in a manner appropriate to their current level of skills. We propose to investigate exploration, the learning of multiple skills, and the construction of higher-level representations together: learning multiple skills facilitates construction of higher-level representations, and when an agent proposes a new higher level representation, it then has natural autonomous motivation for explorations to improve, extend, and validate the representation it has proposed. The main objective of this work package is to develop algorithms and underlying theory for autonomously motivated exploration and skill acquisition. Firstly, we will develop appropriate notions of intrinsic rewards and autonomously motivated exploration, accompanied by behavioural algorithms derived from these notions. Secondly, we will develop skill-based representations of complex domains which allow to compose previously learned skills into new skills.

1. Autonomously motivated exploration. We will develop and evaluate models for intrinsic motivation and curiosity driven exploration in respect to their ability of effecting future behaviour of the learning agent. Useful autonomous exploration should enable the agent to quickly adapt to new tasks and perform them efficiently. We will formally link appropriate notions of intrinsic reward to the overall behaviour of the agent.

2. Skill acquisition. Skill acquisition allows an agent to perform more complex tasks by composing previously learned skills. A skill is the agent’s ability to perform a certain subtask or reach a certain state in a reliable manner. We propose that an intelligent agent should construct a skill-based understanding of its environment; in the sense that it should have (and/or build) a representation of what it knows about how to achieve goals and what skills to apply. We will develop algorithms for constructing such skill-based representations of the environment.