Sample Efficient On-line Learning of Optimal Dialogue Policies with Kalman Temporal Differences

Abstract : Designing dialog policies for voice-enabled interfaces is a tailoring job that is most often left to natural language processing experts. This job is generally redone for every new dialog task because cross-domain transfer is not possible. For this reason, machine learning methods for dialog policy optimization have been investigated during the last 15 years. Especially, reinforcement learning (RL) is now part of the state of the art in this domain. Standard RL methods require to test more or less random changes in the policy on users to assess them as improvements or degradations. This is called on policy learning. Nevertheless, it can result in system behaviors that are not acceptable by users. Learning algorithms should ideally infer an optimal strategy by observing interactions generated by a non-optimal but acceptable strategy, that is learning off-policy. In this contribution, a sample-efficient, online and off-policy reinforcement learning algorithm is proposed to learn an optimal policy from few hundreds of dialogues generated with a very simple handcrafted policy.
Document type :
Conference papers
Complete list of metadatas

Cited literature [20 references]  Display  Hide  Download

https://hal-supelec.archives-ouvertes.fr/hal-00618252
Contributor : Sébastien van Luchene <>
Submitted on : Thursday, September 1, 2011 - 11:30:43 AM
Last modification on : Wednesday, July 31, 2019 - 4:18:02 PM
Long-term archiving on : Sunday, December 4, 2016 - 5:17:24 PM

File

IJCAI_2011_OPMGSC.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00618252, version 1

Collections

Citation

Olivier Pietquin, Matthieu Geist, Senthilkumar Chandramohan. Sample Efficient On-line Learning of Optimal Dialogue Policies with Kalman Temporal Differences. IJCAI 2011, Jul 2011, Barcelona, Spain. pp.1878-1883. ⟨hal-00618252⟩

Share

Metrics

Record views

241

Files downloads

268