Abstract—In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new exploration/exploitation policy (EEP) is proposed. Unlike existing EEPs such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. The simulated experiments, using a ball-catching problem, show that combining this policy with Qlearning is more effective and efficient compared with ε-greedy in stationary environments and UQ-learning in non stationary environments.

Index Terms—Autonomous mobile robot, exploration/exploitation policy, large, unknown and dynamic environment, reinforcement learning.

The authors are with National School of Computer Science, University of Manouba, Tunisia.

Cite:Wiem Zemzem, Moncef Tagina, "A Novel Exploration/Exploitation Policy Accelerating Learning in Both Stationary and Non-Stationary Environment Navigation Tasks," International Journal of Computer and Electrical Engineering vol. 7, no. 3, pp. 149-158, 2015.