Structured Classification for Inverse Reinforcement Learning

Edouard Klein 1 Bilal Piot 1 Matthieu Geist 1 Olivier Pietquin 1
1 IMS - Equipe Information, Multimodalité et Signal
UMI2958 - Georgia Tech - CNRS [Metz], SUPELEC-Campus Metz
Abstract : This paper addresses the Inverse Reinforcement Learning (IRL) problem which is a particular case of learning from demonstrations. The IRL framework assumes that an expert, demonstrating a task, is acting optimally with respect to an unknown reward function to be discovered. Unlike most of existing IRL algorithms, the proposed approach doesn't require any of the following: complete trajectories from the expert, a generative model of the environment, the knowledge of the transition probabilities, the ability to repeatedly solve the forward Reinforcement Learning (RL) problem, the expert's policy anywhere in the state space. Using a classi cation approach in which the structure of the underlying Markov Decision Process (MDP) is implicitly injected, we end-up with an e cient subgradient descent-based algorithm. In addition, only a small amount of expert demonstrations (not even in the form of trajectories but simple transitions) is required. Keywords: inverse reinforcement learning, structured multi-class classi cation
Document type :
Conference papers
Complete list of metadatas

https://hal-supelec.archives-ouvertes.fr/hal-00749524
Contributor : Sébastien van Luchene <>
Submitted on : Wednesday, November 7, 2012 - 4:27:20 PM
Last modification on : Wednesday, July 31, 2019 - 4:18:03 PM

Identifiers

  • HAL Id : hal-00749524, version 1

Collections

Citation

Edouard Klein, Bilal Piot, Matthieu Geist, Olivier Pietquin. Structured Classification for Inverse Reinforcement Learning. EWRL 2012, Jun 2012, Edinburgh, United Kingdom. pp.1-14. ⟨hal-00749524⟩

Share

Metrics

Record views

235