Statistical Linearization for Value Function Approximation in Reinforcement Learning

Abstract : Reinforcement learning (RL) is a machine learning answer to the optimal control problem. It consists in learning an optimal control policy through interactions with the system to be controlled, the quality of this policy being quantified by the so-called value function. An important RL subtopic is to approximate this function when the system is too large for an exact representation. This paper presents statistical-linearization-based approaches to estimate such functions. Compared to more classical approaches, this allows considering nonlinear parameterizations as well as the Bellman optimality operator, which induces some differentiability problems. Moreover, the statistical point of view adopted here allows considering colored observation noise models instead of the classical white one; in RL, this can provide useful.
Document type :
Conference papers
Complete list of metadatas

https://hal-supelec.archives-ouvertes.fr/hal-00554324
Contributor : Sébastien van Luchene <>
Submitted on : Monday, January 10, 2011 - 3:59:24 PM
Last modification on : Thursday, March 29, 2018 - 11:06:04 AM

Identifiers

  • HAL Id : hal-00554324, version 1

Collections

Citation

Matthieu Geist. Statistical Linearization for Value Function Approximation in Reinforcement Learning. NIPS Workshop on Learning and Planning from Batch Time Series Data (OPT 2010), Dec 2010, Vancouver, Canada. pp.1-6. ⟨hal-00554324⟩

Share

Metrics

Record views

33