Mahmudunnobe, Hasan, Raja and Hasan (2021) trained a supervised random-forest classifier on published membership catalogs of nine open clusters and used Gaia DR2 proper motions and parallaxes to assign memberships with roughly 90% precision, often doubling published member counts and revealing sub-clumps beyond nominal cluster radii. The authors explicitly flagged what remains unknown: how training-catalog quality and contamination propagate into the assigned labels, how overlapping sub-clumps can be quantitatively decomposed, and whether the nine-cluster empirical correlations generalize; a further gap — the model was tuned for maximum precision while the assigned membership probabilities were never characterized against threshold-dependent error — is implicit in the reported method rather than flagged by the authors. The problem — a validated reliability characterization of machine-learning membership determination for open clusters — remains open.

Background

An open cluster is a gravitationally bound group of a few hundred to a few thousand stars born from the same molecular cloud, so its members share age, distance, and chemical composition; this makes clusters primary laboratories for stellar evolution, the initial mass function, and Galactic-disk history. Almost every cluster science product (distance, extinction, mass function, dynamical state) inherits its error budget from the membership list: which stars in a crowded Galactic field truly belong to the cluster. The Gaia satellite, whose second data release (DR2) delivered sub-milliarcsecond astrometry — sky position, annual proper motion (transverse angular velocity, in mas/yr), and parallax (the apparent annual shift that measures distance, in mas) — for over a billion sources, transformed membership determination from a few dozen bright stars per cluster to thousands of candidate members per field, at the price of large, structured field-star contamination.

Classical membership methods fit the cluster's proper-motion/parallax distribution against the smooth Galactic field probabilistically. The machine-learning alternative studied here is the random forest (RF): an ensemble of decision trees whose combined vote gives a class-membership probability. As a supervised method, RF must be trained on labeled examples — here the published member lists of Cantat-Gaudin et al. (2018) for nine clusters — and it then classifies all Gaia sources in a search region as member or field star. Mahmudunnobe et al. (2021) did exactly this, selecting their model for maximum precision (the fraction of assigned members that are true members, around 90%) via randomized 5-fold cross-validation over a hyperparameter grid, drawing the field-star (non-member) training samples from concentric rings outside each cluster's search region, and then classifying every Gaia source in the search region as member or field star, with a membership probability given by the fraction of trees voting for membership. They roughly doubled the member counts, found members in outer regions, and identified additional sub-clumps — spatially and kinematically distinct stellar groupings that overlap in sky position and proper-motion space — in NGC 2244, NGC 2264 and NGC 6913; because the clumps overlap, they could not assign sizes or radii to individual clumps and reported only clump centers plus R50R_{50}, the radius containing half the members.

The authors themselves identified several unresolved issues, and the reported method leaves one further gap. First, the method's dependence on its training set: 'a poor training set will yield unreliable results and hence is a major caveat'; the effect of using a different training catalog was never quantified, nor was that of an unrepresentative training set — one whose labeled members, even when correctly labeled, do not span the full range of magnitudes, proper motions and parallaxes covered by the cluster population — nor that of contamination in the training labels (field stars mislabeled as members, or missed members). Second, the empirical correlations reported for the nine clusters (for example, a correlation of 0.97 between training-set size and number of new members found, and an apparent anti-correlation between precision and search radius) were described by the authors as 'mere indicators' awaiting larger, more diverse samples. Third, no robust quantitative method exists to decompose overlapping sub-clumps in position and proper-motion space into individual clumps with well-defined sizes and membership fractions. A further gap is implicit in the method rather than stated by the authors: tuning for precision leaves the complementary error uncharacterized — completeness (the fraction of true members recovered) versus contamination (the fraction of the resulting list that is field stars) as a function of the assigned membership-probability threshold, and its dependence on cluster properties such as distance, age, richness, and local field density, was not assessed. These gaps matter because downstream cluster science (mass functions, tidal tails, substructure interpretation) inherits membership errors directly; the related dynamical question — whether the extra proper-motion clumps are bound substructures or escaped members beyond the tidal radius (the distance at which the Galaxy's tidal field strips cluster stars) — additionally requires dynamical modeling and spectroscopic follow-up that the source study did not attempt. Subsequent membership studies have largely adopted unsupervised or Bayesian families at scale — the original authors' own Gaia DR3 sequels using a Gaussian mixture model (Mahmudunnobe et al. 2024) and DBSCAN (Raja et al. 2024), and the HDBSCAN cluster census (Hunt & Reffert 2023) — which avoids rather than resolves the question of how training-catalog quality and contamination propagate into supervised membership labels.

Problem Statement

Establish a validated reliability characterization of supervised random-forest membership determination for open-cluster stars from Gaia astrometry (proper motions and parallax), as trained on published member catalogs, by answering three linked questions toward one objective — determining when machine-learning-derived member lists (and the sub-structures they reveal) can be trusted for star-cluster science. All three of the following questions are required to establish this characterization, and answering any proper subset of them does not resolve it:

  1. Training-set sensitivity: quantify how the choice of training catalog, its representativeness, and its contamination (field stars mislabeled as members, or true members missing from the labels) propagate into the assigned membership labels and probabilities of the full field population; this requires characterizing the resulting completeness versus contamination as a function of the assigned membership-probability threshold, not only at the single precision-optimized operating point.

  2. Overlapping sub-clumps: develop a robust, quantitative decomposition of detected substructures that overlap in sky position and proper-motion space, yielding sizes or radii and membership fractions for the individual clumps, replacing the descriptive center-plus-R50R_{50} summaries that the source study was limited to.

  3. Generality: determine whether the specific behaviours reported for the source nine clusters (the correlation between training-set size and the number of recovered members, the anti-correlation between precision and search radius, the roughly 90% precision level, the roughly doubled member counts, and the outer-region member and sub-clump detections) hold across at least one hundred open clusters, an order of magnitude more than the source sample, spanning at least the distance, age, richness and local field-density ranges of those nine.

References

  1. Md Mahmudunnobe, Priya Hasan, Mudasir Raja, S. N. Hasan (2021), 'Membership of stars in open clusters using random forest with Gaia data', The European Physical Journal Special Topics 230, 2177–2191. DOI 10.1140/epjs/s11734-021-00205-x, https://doi.org/10.1140/epjs/s11734-021-00205-x (preprint arXiv:2103.05826)
  2. T. Cantat-Gaudin, C. Jordi, A. Vallenari, A. Bragaglia, L. Balaguer-Núñez, C. Soubiran, D. Bossini, A. Moitinho, A. Castro-Ginard, A. Krone-Martins, L. Casamiquela, R. Sordo, R. Carrera (2018), 'A Gaia DR2 view of the open cluster population in the Milky Way', Astronomy & Astrophysics 618, A93. DOI 10.1051/0004-6361/201833476, https://doi.org/10.1051/0004-6361/201833476 (training-catalog source for the source study)
  3. T. Cantat-Gaudin, F. Anders (2020), 'Clusters and mirages: cataloguing stellar aggregates in the Milky Way', Astronomy & Astrophysics 633, A99. DOI 10.1051/0004-6361/201936691, https://doi.org/10.1051/0004-6361/201936691
  4. Manan Agarwal, Khushboo K Rao, Kaushar Vaidya, Souradeep Bhattacharya (2021), 'ML-MOC: Machine Learning (kNN and GMM) based Membership Determination for Open Clusters', Monthly Notices of the Royal Astronomical Society 502, 2582–2599. DOI 10.1093/mnras/stab118, https://doi.org/10.1093/mnras/stab118
  5. Karl Jaehnig, Jonathan Bird, Kelly Holley-Bockelmann (2021), 'Membership Lists for 431 Open Clusters in Gaia DR2 Using Extreme Deconvolution Gaussian Mixture Models', The Astrophysical Journal 923, 129. DOI 10.3847/1538-4357/ac1d51, https://doi.org/10.3847/1538-4357/ac1d51
  6. P. A. B. Galli, H. Bouy, J. Olivares, N. Miret-Roig, L. M. Sarro, D. Barrado, A. Berihuete (2021), 'χ1 Fornacis cluster DANCe. Census of stars, structure, and kinematics of the cluster with Gaia-EDR3', Astronomy & Astrophysics 654, A122. DOI 10.1051/0004-6361/202141366, https://doi.org/10.1051/0004-6361/202141366
  7. Emily L. Hunt, Sabine Reffert (2021), 'Improving the open cluster census. I. Comparison of clustering algorithms applied to Gaia DR2 data', Astronomy & Astrophysics 646, A104. DOI 10.1051/0004-6361/202039341, https://doi.org/10.1051/0004-6361/202039341
  8. Emily L. Hunt, Sabine Reffert (2023), 'Improving the open cluster census. II. An all-sky cluster catalogue with Gaia DR3', Astronomy & Astrophysics 673, A114. DOI 10.1051/0004-6361/202346285, https://doi.org/10.1051/0004-6361/202346285 (preprint arXiv:2303.13424)
  9. Md Mahmudunnobe, P. Hasan, M. Raja, Md Saifuddin, S. N. Hasan (2024), 'Using GMM in open cluster membership: An insight', Astronomy and Computing 46, 100792. DOI 10.1016/j.ascom.2024.100792, https://doi.org/10.1016/j.ascom.2024.100792 (preprint arXiv:2401.10802)
  10. M. Raja, P. Hasan, Md Mahmudunnobe, Md Saifuddin, S. N. Hasan (2024), 'Membership determination in open clusters using the DBSCAN Clustering Algorithm', Astronomy and Computing 47, 100826. DOI 10.1016/j.ascom.2024.100826, https://doi.org/10.1016/j.ascom.2024.100826 (preprint arXiv:2404.10477)
  11. Emily L. Hunt, Sabine Reffert (2024), 'Improving the open cluster census. III. Using cluster masses, radii, and dynamics to create a cleaned open cluster catalogue', Astronomy & Astrophysics 686, A42. DOI 10.1051/0004-6361/202348662, https://doi.org/10.1051/0004-6361/202348662
  12. Mohammad Noormohammadi, Mehdi Khakian Ghomi, Atefeh Javadi (2024), 'Detection of open cluster members inside and beyond tidal radius by machine learning methods based on Gaia DR3', Monthly Notices of the Royal Astronomical Society 532, 622–642. DOI 10.1093/mnras/stae1448, https://doi.org/10.1093/mnras/stae1448
  13. J. Gregorio-Hetem, A. Hetem (2024), 'Structural properties of subgroups of stars associated with open clusters', Monthly Notices of the Royal Astronomical Society 533, 1782–1799. DOI 10.1093/mnras/stae1869, https://doi.org/10.1093/mnras/stae1869 (preprint arXiv:2408.00140)
  14. Emily L. Hunt, Tristan Cantat-Gaudin, Friedrich Anders, Sagar Malhotra, Lorenzo Spina, Alfred Castro-Ginard, Lorenzo Cavallo (2026), 'The selection function of the Gaia DR3 open cluster census', Astronomy & Astrophysics 706, A341. DOI 10.1051/0004-6361/202557781, https://doi.org/10.1051/0004-6361/202557781 (preprint arXiv:2510.18343)
  15. Huanbin Chi, Xuefen Tian, Jingcheng Yang, Feng Wang, Xu Ding, Jing Chen, Yudan Wang (2026), 'Tidal Stripping as the Driver of Mass Segregation in the Young Open Cluster NGC 2422', The Astronomical Journal 171, 153. DOI 10.3847/1538-3881/ae3759, https://doi.org/10.3847/1538-3881/ae3759
  16. Lu Li, Zhengyi Shao, Zhaozhou Li, Xiaoting Fu (2025), 'The MiMO Catalog: Physical Parameters and Stellar Mass Functions of 1,232 Open Clusters from Gaia DR3', arXiv:2510.23374, https://arxiv.org/abs/2510.23374