Some considerations on learning to explore via meta-reinforcement learning

Bradly C. Stadie; Ge Yang; Rein Houthooft; Xi Chen; Yan Duan; Yuhuai Wu; Pieter Abbeel; Ilya Sutskever

Some considerations on learning to explore via meta-reinforcement learning

Bradly C. Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, Ilya Sutskever

Statistics and Data Science

Research output: Contribution to journal › Conference article › peer-review

11 Scopus citations

Abstract

We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learning algorithms: E-MAML and E-RL². Results are presented on a new environment we call 'Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning. Further results are presented on a set of maze environments. We show E-MAML and E-RL² deliver better performance than baseline algorithms on both tasks.

Original language	English (US)
Pages (from-to)	9280-9290
Number of pages	11
Journal	Advances in Neural Information Processing Systems
Volume	2018-December
State	Published - 2018
Event	32nd Conference on Neural Information Processing Systems, NeurIPS 2018 - Montreal, Canada Duration: Dec 2 2018 → Dec 8 2018

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

@article{7c18eadadc134f4cad60a2eaab1dd233,

title = "Some considerations on learning to explore via meta-reinforcement learning",

abstract = "We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learning algorithms: E-MAML and E-RL2. Results are presented on a new environment we call 'Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning. Further results are presented on a set of maze environments. We show E-MAML and E-RL2 deliver better performance than baseline algorithms on both tasks.",

author = "Stadie, {Bradly C.} and Ge Yang and Rein Houthooft and Xi Chen and Yan Duan and Yuhuai Wu and Pieter Abbeel and Ilya Sutskever",

note = "Funding Information: This work was supported in part by ONR PECASE N000141612723 and by AWS and GCE compute credits. Publisher Copyright: {\textcopyright} 2018 Curran Associates Inc.All rights reserved.; 32nd Conference on Neural Information Processing Systems, NeurIPS 2018 ; Conference date: 02-12-2018 Through 08-12-2018",

year = "2018",

language = "English (US)",

volume = "2018-December",

pages = "9280--9290",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

}

TY - JOUR

T1 - Some considerations on learning to explore via meta-reinforcement learning

AU - Stadie, Bradly C.

AU - Yang, Ge

AU - Houthooft, Rein

AU - Chen, Xi

AU - Duan, Yan

AU - Wu, Yuhuai

AU - Abbeel, Pieter

AU - Sutskever, Ilya

PY - 2018

Y1 - 2018

N2 - We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learning algorithms: E-MAML and E-RL2. Results are presented on a new environment we call 'Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning. Further results are presented on a set of maze environments. We show E-MAML and E-RL2 deliver better performance than baseline algorithms on both tasks.

AB - We interpret meta-reinforcement learning as the problem of learning how to quickly find a good sampling distribution in a new environment. This interpretation leads to the development of two new meta-reinforcement learning algorithms: E-MAML and E-RL2. Results are presented on a new environment we call 'Krazy World': a difficult high-dimensional gridworld which is designed to highlight the importance of correctly differentiating through sampling distributions in meta-reinforcement learning. Further results are presented on a set of maze environments. We show E-MAML and E-RL2 deliver better performance than baseline algorithms on both tasks.

UR - http://www.scopus.com/inward/record.url?scp=85064808318&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064808318&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85064808318

SN - 1049-5258

VL - 2018-December

SP - 9280

EP - 9290

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

T2 - 32nd Conference on Neural Information Processing Systems, NeurIPS 2018

Y2 - 2 December 2018 through 8 December 2018

ER -

Some considerations on learning to explore via meta-reinforcement learning

Abstract

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this