The 2019 Bayesian finite mixture framework of Wang, Yabes and Chang clusters mixed-type (continuous and categorical) data with limits-of-detection censoring and spike-and-slab variable selection, but assumes local independence of variables given cluster membership and Gaussian conditional distributions, with costly MCMC estimation. Later work resolved major parts of that agenda: the same group's 2026 BFMM adds eigen-structured covariance models for the continuous block and robust likelihood-based censoring imputation, and a 2025 CAVI algorithm makes mixed-data mixture inference scalable with asymptotic guarantees — yet BFMM still assumes local independence for categorical variables and Gaussian margins with expensive sampling, while the variational algorithm has no variable selection or censoring treatment. The open problem is the surviving core of this research program: a single validated framework that simultaneously models within-cluster dependence involving categorical variables, accommodates non-Gaussian within-cluster distributions, handles limits-of-detection censoring robustly at high censoring proportions, preserves calibrated variable-importance estimates for both variable types, and achieves computationally practical inference with uncertainty quantification.

Background

Model-based clustering represents data from a heterogeneous population as a finite mixture model (FMM): the overall distribution is a weighted sum of GG component distributions, each governed by cluster-specific parameters, and each observation is assigned probabilistically to a component. The motivating application is the clustering of clinical datasets — electronic health records and trial data — where the measured variables are of mixed type: some continuous (laboratory biomarkers) and some categorical (demographic groups, comorbidity flags). Moreover, many biomarkers are subject to limits of detection (LOD): values below a lower detection threshold, or above an upper one, are recorded only as lying beyond the threshold. This is left or right censoring of a continuous variable and is common in immunoassay and chemistry panels.

Two structural devices make mixed-type FMMs tractable, and both are at issue in this problem. The first is local independence: conditional on cluster membership, all variables are mutually independent, so the component density factorizes into univariate terms — Gaussian for continuous variables, multinomial for categorical ones. This product form keeps the number of parameters linear in the number of variables, but it rules out within-cluster association: two biomarkers that co-vary within patient subgroups, or a biomarker whose distribution shifts together with a categorical variable inside a cluster, are modeled as independent. The second device is the spike-and-slab prior for variable selection: for each variable, a mixture of a "spike" prior concentrated at a cluster-independent parameter value (the variable is irrelevant to clustering) and a diffuse "slab" prior (the variable is relevant), from which posterior variable-importance weights on a 0–1 scale can be derived; Wang, Yabes and Chang (2019) extended this construction from continuous to categorical variables via a spike-at-the-marginal Dirichlet prior.

Estimation can proceed by the EM algorithm (fast point estimation, but sensitive to initialization and without posterior uncertainty), by Markov chain Monte Carlo (MCMC) such as Gibbs sampling (full posterior uncertainty, higher cost), or by coordinate-ascent variational inference (CAVI), which approximates the posterior within a factorized family and scales much better. For dependence among continuous variables within clusters, the covariance matrix of each Gaussian component can be constrained through eigenvalue decompositions; standard choices from the parsimonious-Gaussian-mixture taxonomy include EEI (one shared diagonal covariance), EEE (one shared full covariance), and VVV (a full covariance per cluster).

Existing frameworks each cover part of the ground. clustMD (McParland and Gormley, 2016) models mixed types by letting a latent Gaussian mixture generate the observed variables, capturing dependence through the latent covariance, but uses EM point estimation and offers no variable selection or censoring treatment. VarSelLCM (Marbac and Sedki, 2018) performs variable selection for mixed-type clustering, still under local independence and via EM. The Gaussian-copula mixtures of Marbac, Biernacki and Vandewalle (2017) model intra-component dependence with correlation in the Gaussian-copula sense and Gaussian, Poisson, or ordinal multinomial margins — but not nominal (unordered) categorical variables, and without variable selection or censoring. Storlie et al. (2017) combine a Dirichlet-process mixture with variable selection for mixed types via latent continuous variables, again without censoring. The 2019 framework of Wang, Yabes and Chang that generated this problem performs Bayesian Gibbs estimation with spike-and-slab selection for both continuous and categorical variables and an embedded truncated-normal sampling step for LOD-censored biomarkers, but assumes local independence and Gaussian conditionals, and is slower than EM procedures; its own simulations show the censored-data treatment losing its advantage at high censoring proportions, where the authors' two-step HyDaP algorithm with ad-hoc imputation is more robust. Its same-group 2026 successor, BFMM (Wang, Wang, Yabes and Chang), adds EEI/EEE/VVV covariance structures for the continuous block (within-cluster dependence among continuous variables), likelihood-based imputation of censored values whose accuracy is reported to hold at high censoring proportions, and a data-driven calibration of the slab/spike variance ratio — while still assuming local independence for categorical variables, Gaussian within-component margins for continuous variables, and paying a substantial MCMC cost, especially under the VVV structure. Outside this program, Wang, Bennett, Lhoste and Filippi (2025) derived a CAVI algorithm for mixed-data mixtures with Gaussian continuous blocks and conditionally independent categoricals, with uncertainty quantification and asymptotic theory, but with no variable selection and no censoring treatment. Reviews of variable selection in model-based clustering (Fop and Murphy, 2018) treat the local-independence setting as standard.

No framework that simultaneously relaxes local independence for categorical variables, accommodates non-Gaussian within-cluster margins, handles LOD censoring robustly, preserves calibrated variable importance, and attains practical computational cost is available; attaining that conjunction is the objective.

Problem Statement

Develop and validate a single model-based clustering framework for mixed-type data — continuous and nominal categorical variables, where continuous variables may be left- or right-censored by limits of detection — that simultaneously achieves all of the following:

(1) Within-cluster dependence across variable types. The component density must allow dependence not only among continuous variables (as BFMM's EEI/EEE/VVV structures already do) but also involving categorical variables: categorical–categorical and categorical–continuous association within clusters must be expressible.

(2) Flexible within-cluster margins. The conditional distributions of continuous variables given cluster membership must accommodate at least skewness and heavy tails (e.g., skew-normal or skew-tt families) rather than being restricted to the Gaussian family.

(3) Robust censoring handling. Censored values must be handled by a principled mechanism whose clustering and selection accuracy does not deteriorate at high censoring proportions — the regime where the 2019 embedded truncated-normal approach lost its advantage to ad-hoc imputation.

(4) Calibrated variable importance. Per-variable importance (e.g., posterior spike-and-slab weights) must be available for both continuous and categorical variables, with demonstrated ability to discriminate relevant from noise variables under the relaxed assumptions (1)–(3), and in a form comparable across variable types.

(5) Practical computation. Inference must be computationally practical for electronic-health-record-scale datasets, with uncertainty quantification.

The five requirements are jointly the objective because they co-occur in the motivating clinical data: a framework satisfying only a subset replicates what existing methods (clustMD, VarSelLCM, Gaussian-copula mixtures, BFMM, CAVI mixed-data mixtures) already provide in pairwise pieces. The objective is a single framework satisfying (1)–(5), validated on mixed-type data (continuous and nominal categorical variables) that carry limits-of-detection censoring and include a subset of clustering-irrelevant variables, demonstrating — against the partial-capability methods enumerated above together with the 2019 truncated-normal treatment and ad-hoc imputation, and, where applicable, the clustMD and Gaussian-copula-mixture dependence models — clustering accuracy, measured by the adjusted Rand index against the known partition, and variable-importance discrimination, measured by the F1 score for recovering the relevant-variable set, both averaged over a grid of censoring proportions that reaches the high-censoring regime in which the 2019 treatment lost its advantage to ad-hoc imputation and neither falling below the best applicable comparator at the highest proportion of that grid, together with full posterior uncertainty and inference that remains practical at electronic-health-record scale (at least 10410^4 subjects and 10210^2 candidate variables, at a runtime no greater than a few hours on a single workstation).

References

  1. Shu Wang, Jonathan G. Yabes, and Chung-Chou H. Chang. A Bayesian Finite Mixture Model with Variable Selection for Data with Mixed-type Variables. arXiv:1905.03680 (2019). https://arxiv.org/abs/1905.03680
  2. Yueting Wang, Shu Wang, Jonathan G. Yabes, and Chung-Chou H. Chang. A Bayesian Finite Mixture Model Approach for Mixed-type Data Clustering and Variable Selection with Censored Biomarkers. arXiv:2603.29316 (2026). https://arxiv.org/abs/2603.29316
  3. Junyang Wang, James Bennett, Victor Lhoste, and Sarah Filippi. Bayesian Variational Inference for Mixed Data Mixture Models. arXiv:2507.16545 (2025). https://arxiv.org/abs/2507.16545
  4. Matthieu Marbac, Christophe Biernacki, and Vincent Vandewalle. Model-based clustering of Gaussian copulas for mixed data. Communications in Statistics – Theory and Methods 46(23):11635–11656 (2017). DOI: 10.1080/03610926.2016.1277753. https://doi.org/10.1080/03610926.2016.1277753
  5. Damien McParland and Isobel Claire Gormley. Model based clustering for mixed data: clustMD. Advances in Data Analysis and Classification 10(2):155–169 (2016). DOI: 10.1007/s11634-016-0238-x. https://doi.org/10.1007/s11634-016-0238-x
  6. Matthieu Marbac and Mohammed Sedki. VarSelLCM: an R/C++ package for variable selection in model-based clustering of mixed-data with missing values. Bioinformatics 35(7):1255–1257 (2018). DOI: 10.1093/bioinformatics/bty786. https://doi.org/10.1093/bioinformatics/bty786
  7. Michael Fop and Thomas Brendan Murphy. Variable selection methods for model-based clustering. Statistics Surveys 12:18–65 (2018). DOI: 10.1214/18-ss119. https://doi.org/10.1214/18-ss119
  8. Curtis Storlie, Scott Myers, Slavica Katusic, Amy Weaver, Robert Voigt, Robert Colligan, Paul Croarkin, Ruth Stoeckel, and John Port. Clustering and Variable Selection in the Presence of Mixed Variable Types and Missing Data. arXiv:1703.08741 (2017). https://arxiv.org/abs/1703.08741
  9. Shu Wang, Jonathan G. Yabes, and Chung-Chou H. Chang. Hybrid Density- and Partition-based Clustering Algorithm for Data with Mixed-type Variables. arXiv:1905.02257 (2019); published as Journal of Data Science 19:15–36 (2021). DOI: 10.6339/21-JDS996. https://arxiv.org/abs/1905.02257
  10. Matthieu Marbac, Christophe Biernacki, and Vincent Vandewalle. Model-based clustering for conditionally correlated categorical data. arXiv:1401.5684 (2014). https://arxiv.org/abs/1401.5684