Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping

Hédy Attouch (1, 2) , Zaki Chbani (3, 4) , Jalal M. Fadili (5, 6, 7, 8) , Hassan Riahi (3, 4)

1 IMAG - Institut Montpelliérain Alexander Grothendieck
2 UM - Université de Montpellier
3 UCA - Université Cadi Ayyad [Marrakech]
4 Faculté des Sciences Semlalia [Marrakech]
5 Equipe Image - Laboratoire GREYC - UMR6072
6 NU - Normandie Université
7 ENSICAEN - École Nationale Supérieure d'Ingénieurs de Caen
8 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen

In a Hilbert space setting, for convex optimization, we show the convergence of the iterates to optimal solutions for a class of accelerated first-order algorithms. They can be interpreted as discrete temporal versions of an inertial dynamic involving both viscous damping and Hessian-driven damping. The asymptotically vanishing viscous damping is linked to the accelerated gradient method of Nesterov while the Hessian driven damping makes it possible to significantly attenuate the oscillations. By treating the Hessian-driven damping as the time derivative of the gradient term, this gives, in discretized form, first-order algorithms. These results complement the previous work of the authors where it was shown the fast convergence of the values, and the fast convergence towards zero of the gradients. KEYWORDS Convergence of iterates; Hessian driven damping; inertial optimization algorithms; Nesterov accelerated gradient method; time rescaling.

Mots clés: Convergence of iterates; Hessian driven damping; inertial optimization algorithms; Nesterov accelerated gradient method; time rescaling

Domaines: Optimisation et contrôle [math.OC]; Analyse numérique [math.NA]

Hédy Attouch, Zaki Chbani, Jalal M. Fadili, Hassan Riahi. Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping. Optimization, 2021, ⟨10.1080/02331934.2021.2009828⟩. ⟨hal-03285272⟩