Learning to select state machines using expert advice on an autonomous robot

Brenna Argall*, Brett Browning, Manuela Veloso

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

Hierarchical state machines have proven to be a powerful tool for controlling autonomous robots due to their flexibility and modularity. For most real robot implementations, however, it is often the case that the control hierarchy is hand-coded. As a result, the development process is often time intensive and error prone. In this paper, we explore the use of an experts learning approach, based on Auer and colleagues' Exp3 [1], to help overcome some of these limitations. In particular, we develop a modified learning algorithm, which we call rExp3, that exploits the structure provided by a control hierarchary by treating each state machine as an 'expert'. Our experiments validate the performance of rExp3 on a real robot performing a task, and demonstrate that rExp3 is able to quickly learn to select the best state machine expert to execute. Through our investigations in these environments, we identify a need for faster learning recovery when the relative performances of experts reorder, such as in response to a discrete environment change. We introduce a modified learning rule to improve the recovery rate in these situations and demonstrate through simulation experiments that rExp3 performs as well or better than Exp3 under such conditions.

Original languageEnglish (US)
Title of host publication2007 IEEE International Conference on Robotics and Automation, ICRA'07
Pages2124-2129
Number of pages6
DOIs
StatePublished - Nov 27 2007
Event2007 IEEE International Conference on Robotics and Automation, ICRA'07 - Rome, Italy
Duration: Apr 10 2007Apr 14 2007

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Other

Other2007 IEEE International Conference on Robotics and Automation, ICRA'07
CountryItaly
CityRome
Period4/10/074/14/07

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Learning to select state machines using expert advice on an autonomous robot'. Together they form a unique fingerprint.

Cite this