Volume 7 Number 3 (Jun. 2015)
Home > Archive > 2015 > Volume 7 Number 3 (Jun. 2015) >
IJCEE 2015 Vol.7(3): 149-158 ISSN: 1793-8163
DOI: 10.17706/IJCEE.2015.7.3.149-158

A Novel Exploration/Exploitation Policy Accelerating Learning in Both Stationary and Non-Stationary Environment Navigation Tasks

Wiem Zemzem, Moncef Tagina
Abstract—In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new exploration/exploitation policy (EEP) is proposed. Unlike existing EEPs such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. The simulated experiments, using a ball-catching problem, show that combining this policy with Qlearning is more effective and efficient compared with ε-greedy in stationary environments and UQ-learning in non stationary environments.

Index Terms—Autonomous mobile robot, exploration/exploitation policy, large, unknown and dynamic environment, reinforcement learning.

The authors are with National School of Computer Science, University of Manouba, Tunisia.

Cite:Wiem Zemzem, Moncef Tagina, "A Novel Exploration/Exploitation Policy Accelerating Learning in Both Stationary and Non-Stationary Environment Navigation Tasks," International Journal of Computer and Electrical Engineering vol. 7, no. 3, pp. 149-158, 2015.

General Information

ISSN: 1793-8163 (Print)
Abbreviated Title: Int. J. Comput. Electr. Eng.
Frequency: Quarterly
Editor-in-Chief: Prof. Yucong Duan
Abstracting/ Indexing: EI (INSPEC, IET), Ulrich's Periodicals Directory, Google Scholar, EBSCO, ProQuest, and Electronic Journals Library
E-mail: ijcee@iap.org

What's New

  • Jun 03, 2019 News!

    IJCEE Vol. 9, No. 2 - Vol. 10, No. 2 have been indexed by EI (Inspec) Inspec, created by the Institution of Engineering and Tech.!   [Click]

  • Dec 11, 2019 News!

    The dois of published papers in Vol 11, No 4 have been validated by Crossref

  • Oct 11, 2019 News!

    IJCEE Vol 11, No 4 is available online now   [Click]

  • Oct 11, 2019 News!

    The dois of published papers in Vol 11, No 1- Vol 11, No 3 have been validated by Crossref

  • Aug 20, 2019 News!

    IJCEE Vol 11, No 3 is available online now   [Click]

  • Read more>>