Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Is interpolation benign for random forests?

Abstract : Statistical wisdom suggests that very complex models, interpolating training data, will be poor at prediction on unseen examples. Yet, this aphorism has been recently challenged by the identification of benign overfitting regimes, specially studied in the case of parametric models: generalization capabilities may be preserved despite model high complexity. While it is widely known that fully-grown decision trees interpolate and, in turn, have bad predictive performances, the same behavior is yet to be analyzed for random forests. In this paper, we study the trade-off between interpolation and consistency for several types of random forest algorithms. Theoretically, we prove that interpolation regimes and consistency cannot be achieved for non-adaptive random forests. Since adaptivity seems to be the cornerstone to bring together interpolation and consistency, we introduce and study interpolating Adaptive Centered Forests, which are proved to be consistent in a noiseless scenario. Numerical experiments show that Breiman's random forests are consistent while exactly interpolating, when no bootstrap step is involved. We theoretically control the size of the interpolation area, which converges fast enough to zero, so that exact interpolation and consistency occur in conjunction.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03560047
Contributor : Claire Boyer Connect in order to contact the contributor
Submitted on : Friday, April 29, 2022 - 4:16:47 PM
Last modification on : Wednesday, June 8, 2022 - 12:50:05 PM

Files

main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03560047, version 2
  • ARXIV : 2202.03688

Citation

Ludovic Arnould, Claire Boyer, Erwan Scornet. Is interpolation benign for random forests?. 2022. ⟨hal-03560047v2⟩

Share

Metrics

Record views

121

Files downloads

41