Provably Convergent Working Set Algorithm for Non-Convex Regularized Regression - Centre de mathématiques appliquées (CMAP) Accéder directement au contenu
Proceedings/Recueil Des Communications Année : 2022

Provably Convergent Working Set Algorithm for Non-Convex Regularized Regression

Résumé

Owing to their statistical properties, non-convex sparse regularizers have attracted much interest for estimating a sparse linear model from high dimensional data. Given that the solution is sparse, for accelerating convergence, a working set strategy addresses the optimization problem through an iterative algorithm by incre-menting the number of variables to optimize until the identification of the solution support. While those methods have been well-studied and theoretically supported for convex regularizers, this paper proposes a working set algorithm for non-convex sparse regularizers with convergence guarantees. The algorithm, named FireWorks, is based on a non-convex reformulation of a recent primal-dual approach and leverages on the geometry of the residuals. Our theoretical guarantees derive from a lower bound of the objective function decrease between two inner solver iterations and shows the convergence to a stationary point of the full problem. More importantly, we also show that convergence is preserved even when the inner solver is inexact, under sufficient decay of the error across iterations. Our experimental results demonstrate high computational gain when using our working set strategy compared to the full problem solver for both block-coordinate descent or a proximal gradient solver.
Fichier principal
Vignette du fichier
activeset_ncvx.pdf (930.97 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02875560 , version 1 (19-06-2020)
hal-02875560 , version 2 (24-06-2020)
hal-02875560 , version 3 (15-10-2020)
hal-02875560 , version 4 (19-10-2021)

Licence

Paternité

Identifiants

Citer

Alain Rakotomamonjy, Rémi Flamary, Gilles Gasso, Joseph Salmon. Provably Convergent Working Set Algorithm for Non-Convex Regularized Regression. International Conference on Artificial Intelligence and Statistics, 2022. ⟨hal-02875560v4⟩
229 Consultations
143 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More