Estimation and imputation in Probabilistic Principal Component Analysis with Missing Not At Random data - Centre de mathématiques appliquées (CMAP) Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Estimation and imputation in Probabilistic Principal Component Analysis with Missing Not At Random data

Résumé

Missing Not At Random (MNAR) values lead to significant biases in the data, since the probability of missingness depends on the unobserved values. They are "not ignorable" in the sense that they often require defining a model for the missing data mechanism, which makes inference or imputation tasks more complex. Furthermore, this implies a strong \textit{a priori} on the parametric form of the distribution. However, some works have obtained guarantees on the estimation of parameters in the presence of MNAR data, without specifying the distribution of missing data \citep{mohan2018estimation, tang2003analysis}. This is very useful in practice, but is limited to simple cases such as self-masked MNAR values in data generated according to linear regression models. We continue this line of research, but extend it to a more general MNAR mechanism, in a more general model of the probabilistic principal component analysis (PPCA), \textit{i.e.}, a low-rank model with random effects. We prove identifiability of the PPCA parameters. We then propose an estimation of the loading coefficients and a data imputation method. They are based on estimators of means, variances and covariances of missing variables, for which consistency is discussed. These estimators have the great advantage of being calculated using only the observed data, leveraging the underlying low-rank structure of the data. We illustrate the relevance of the method with numerical experiments on synthetic data and also on real data collected from a medical register.
Fichier principal
Vignette du fichier
MNAR_PPCA_graphical_2020.pdf (565.61 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02146983 , version 1 (04-06-2019)
hal-02146983 , version 2 (25-07-2019)
hal-02146983 , version 3 (04-06-2020)

Identifiants

Citer

Aude Sportisse, Claire Boyer, Julie Josse. Estimation and imputation in Probabilistic Principal Component Analysis with Missing Not At Random data. NeurIPS, Neural Information Processing Systems, Dec 2020, Vancouver, Canada. ⟨hal-02146983v3⟩
192 Consultations
126 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More