Random Projections: a Remedy for Overfitting Issues in Time Series Prediction with Echo State Networks

Lucie Daubigney 1, 2 Matthieu Geist 3 Olivier Pietquin 2
1 MAIA - Autonomous intelligent machine
Inria Nancy - Grand Est, LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
2 IMS - Equipe Information, Multimodalité et Signal
UMI2958 - Georgia Tech - CNRS [Metz], SUPELEC-Campus Metz
Abstract : Modelling time series is quite a difficult task. The last recent years, reservoir computing approaches have been proven very efficient for such problems. Indeed, thanks to recurrence in the connections between neurons, this approach is a powerful tool to catch and model time dependencies between samples. Yet, the prediction quality often depends on the trade-off between the number of neurons in the reservoir and the amount of training data. Supposedly, the larger the number of neurons, the richer the reservoir of dynamics. However, the risk of overfitting problem appears. Conversely, the lower the number of neurons is, the lower the risk of overfitting problem is but also the poorer the reservoir of dynamics is. We consider here the combination of an echo state network with a projection method to benefit from the advantages of the reservoir computing approach without needing to pay attention to overfitting problems due to a lack of training data.
Document type :
Conference papers
Complete list of metadatas

Cited literature [13 references]  Display  Hide  Download

https://hal-supelec.archives-ouvertes.fr/hal-00869814
Contributor : Sébastien van Luchene <>
Submitted on : Monday, November 6, 2017 - 5:46:50 PM
Last modification on : Wednesday, July 31, 2019 - 4:18:03 PM

File

rp_esn.pdf
Files produced by the author(s)

Identifiers

Citation

Lucie Daubigney, Matthieu Geist, Olivier Pietquin. Random Projections: a Remedy for Overfitting Issues in Time Series Prediction with Echo State Networks. ICASSP 2013, May 2013, Vancouver, Canada. pp.3253-3257, ⟨10.1109/ICASSP.2013.6638259⟩. ⟨hal-00869814⟩

Share

Metrics

Record views

635

Files downloads

199