Volume 7 Number 3 (Jun. 2015)
Home > Archive > 2015 > Volume 7 Number 3 (Jun. 2015) >
IJCEE 2015 Vol.7(3): 149-158 ISSN: 1793-8163
DOI: 10.17706/IJCEE.2015.7.3.149-158

A Novel Exploration/Exploitation Policy Accelerating Learning in Both Stationary and Non-Stationary Environment Navigation Tasks

Wiem Zemzem, Moncef Tagina
Abstract—In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new exploration/exploitation policy (EEP) is proposed. Unlike existing EEPs such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. The simulated experiments, using a ball-catching problem, show that combining this policy with Qlearning is more effective and efficient compared with ε-greedy in stationary environments and UQ-learning in non stationary environments.

Index Terms—Autonomous mobile robot, exploration/exploitation policy, large, unknown and dynamic environment, reinforcement learning.

The authors are with National School of Computer Science, University of Manouba, Tunisia.

Cite:Wiem Zemzem, Moncef Tagina, "A Novel Exploration/Exploitation Policy Accelerating Learning in Both Stationary and Non-Stationary Environment Navigation Tasks," International Journal of Computer and Electrical Engineering vol. 7, no. 3, pp. 149-158, 2015.

General Information

ISSN: 1793-8163
Frequency: Semiyearly
Editor-in-Chief: Prof. Yucong Duan
Abstracting/ Indexing: EI (INSPEC, IET), Ulrich's Periodicals Directory, Google Scholar, EBSCO, Engineering & Technology Digital Library, ProQuest, and Electronic Journals Library
E-mail: ijcee@iap.org

What's New

  • Jul 27, 2017 News!

    IJCEE Vol. 9, No. 1 is available online now.   [Click]

  • Jun 29, 2017 News!

    IJCEE Vol. 7, No. 6 has been indexed by EI (Inspec) Inspec, created by the Institution of Engineering and Tech.!   [Click]

  • Jun 29, 2017 News!

    IJCEE Vol. 7, No. 5 has been indexed by EI (Inspec) Inspec, created by the Institution of Engineering and Tech.!   [Click]

  • Jun 29, 2017 News!

    IJCEE Vol. 7, No. 4 has been indexed by EI (Inspec) Inspec, created by the Institution of Engineering and Tech.!   [Click]

  • Jun 29, 2017 News!

    IJCEE Vol. 7, No. 1 has been indexed by EI (Inspec) Inspec, created by the Institution of Engineering and Tech.!   [Click]

  • Read more>>