Deterministic Approximate EM Algorithm; Application to the Riemann Approximation EM and the Tempered EM - Centre de mathématiques appliquées (CMAP) Accéder directement au contenu
Article Dans Une Revue Algorithms Année : 2022

Deterministic Approximate EM Algorithm; Application to the Riemann Approximation EM and the Tempered EM

Résumé

The Expectation Maximisation (EM) algorithm is widely used to optimise non-convex likelihood functions with latent variables. Many authors modified its simple design to fit more specific situations. For instance, the Expectation (E) step has been replaced by Monte Carlo (MC), Markov Chain Monte Carlo or tempered approximations, etc. Most of the well-studied approximations belong to the stochastic class. By comparison, the literature is lacking when it comes to deterministic approximations. In this paper, we introduce a theoretical framework, with state-of-the-art convergence guarantees, for any deterministic approximation of the E step. We analyse theoretically and empirically several approximations that fit into this framework. First, for intractable E-steps, we introduce a deterministic version of MC-EM using Riemann sums. A straightforward method, not requiring any hyper-parameter fine-tuning, useful when the low dimensionality does not warrant a MC-EM. Then, we consider the tempered approximation, borrowed from the Simulated Annealing literature and used to escape local extrema. We prove that the tempered EM verifies the convergence guarantees for a wider range of temperature profiles than previously considered. We showcase empirically how new non-trivial profiles can more successfully escape adversarial initialisations. Finally, we combine the Riemann and tempered approximations into a method that accomplishes both their purposes.
Fichier principal
Vignette du fichier
main.pdf (977.53 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02513593 , version 1 (20-03-2020)
hal-02513593 , version 2 (18-12-2020)
hal-02513593 , version 3 (03-02-2022)
hal-02513593 , version 4 (02-05-2022)

Licence

Paternité

Identifiants

Citer

Thomas Lartigue, Stanley Durrleman, Stéphanie Allassonnière. Deterministic Approximate EM Algorithm; Application to the Riemann Approximation EM and the Tempered EM. Algorithms, 2022, Stochastic Algorithms and Their Applications, 15 (3), pp.78. ⟨10.3390/a15030078⟩. ⟨hal-02513593v4⟩
353 Consultations
614 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More