Geurts, Engel, Rafii, Blanchet, Suppers, Szymańska, Jansen and Buydens (Chemometrics and Intelligent Laboratory Systems 156, 2016, 231–240) introduced "sustainable fusion", a mid-level data-fusion strategy for high-dimensional multi-platform chemical data that recycles the information conventional fusion discards: after partial-least-squares (PLS) modeling of the predictive part of each data block, the residual (non-predictive) parts of different blocks still share structure, quantified by intraclass correlation (classification) or partial correlation (regression), and a regression using that correlation recovers the discarded information and improves prediction. The method was demonstrated only on two-block problems (a simulation study and beer-bitterness regression), and the authors left its development agenda open: extending the recycling principle to more than two blocks with a principled rule for selecting which predictive–residual pairs to recycle; characterizing the sample-size dependence of the residual-based correlation estimates; quantifying prediction uncertainty; porting the recycling principle to non-PLS fusion frameworks; and applying it to ultra-high-dimensional feature screening. No development of these items is known in the literature to date. The problem — developing the residual-recycling paradigm into a general, statistically validated multi-block methodology — remains open.

Background

In analytical chemistry, chemometrics, and omics sciences, the same samples are routinely measured on several instrumental platforms (e.g., near-infrared and mid-infrared spectra, chromatograms, mass spectra), giving several high-dimensional data blocks with few samples. Data fusion is the joint chemometric analysis of such blocks to improve classification or regression. Three classical levels are distinguished: low-level fusion concatenates the raw blocks; mid-level fusion extracts a low-dimensional feature set (typically latent-variable scores) from each block and combines the features; high-level fusion combines the outputs of separate per-block models. Mid- and high-level fusion deliberately discard the part of each block that does not predict the response, which removes noise but can also destroy information. Partial least squares (PLS) is the workhorse latent-variable method: it regresses a response on projections (scores) of the predictor block chosen to maximize covariance with the response, and deflation is the step of subtracting the fitted predictive part, leaving residuals. Geurts et al. (2016) observed that this discarded residual information is not necessarily useless: when several blocks are considered together, the discarded part of one block can correlate with the retained (predictive) part of another. They call such cross-block correlation between discarded and retained parts, computed within classes, the intraclass correlation (in the classification setting); in the regression setting the analogous quantity is a partial correlation, i.e., a correlation between two variable groups computed after controlling for the retained predictive part. Because this information appears only when multiple blocks are analyzed jointly, they refer to it as the multivariate advantage. Their "sustainable fusion" strategy recycles the residual information via a regression employing these correlations, instead of discarding it — "sustainable" in the sense that all measured information is used. The authors demonstrated the strategy on two data blocks only: a comprehensive simulation study showing classification gains over conventional low-, mid- and high-level fusion in many scenarios of shared (communal) information between blocks, and a real regression example predicting the bitterness of beers. They explicitly framed the further development of the paradigm as open: the paper suggests, but does not develop, an extension to more than two blocks — where the number of candidate predictive–residual pairs and cross-residual correlations grows combinatorially and a principled selection rule is needed — as well as systematic study of how sample size affects the reliability of the intraclass/partial correlation estimates obtained from deflated matrices (a critical concern in small-sample omics settings), characterization of prediction uncertainty (confidence intervals for regression estimates, calibrated classification probabilities), incorporation of the recycling principle into fusion frameworks other than PLS-based mid-level fusion, and use of the recycling idea in feature screening for ultra-high-dimensional data, where variables would be selected for their intraclass correlations as well as their marginal association with the response.

Problem Statement

Develop the residual-recycling ("sustainable") fusion paradigm of Geurts et al. (2016) from a two-block proof of concept into a general, statistically validated multi-block methodology for high-dimensional data fusion. The overarching objective is to establish whether, when, and how reliably recycling cross-block residual information via intraclass/partial correlation improves prediction when more than two data blocks are available. Concretely, this objective comprises the following open methodological items: (1) a principled extension to an arbitrary number of data blocks, including a defensible rule for selecting which predictive–residual block pairs to recycle when the number of candidate cross-block residual correlations grows combinatorially; (2) characterization of how sample size affects the reliability of the intraclass and partial correlation estimates obtained from deflated (residual) matrices and the resulting prediction performance, particularly in the small-sample settings typical of omics data; and (3) quantification of the uncertainty of the method's predictions, such as confidence intervals for regression estimates and calibrated classification probabilities. The source also proposed two further extensions of the recycling principle: (4) incorporating it into data-fusion frameworks other than PLS-based mid-level fusion, and (5) applying it to feature screening for ultra-high-dimensional data, where variables are selected for their intraclass correlations with other blocks as well as their marginal association with the response. Items (1)–(3) define the core methodological objective, whereas items (4)–(5) broaden the paradigm's generality. The evidence may equally support a negative or only conditionally positive conclusion about whether recycling improves multi-block prediction, and the reliability of such a conclusion rests on the accuracy of the residual-based correlation estimates (item 2) and on the characterization of prediction uncertainty (item 3).

References

  1. B. P. Geurts, J. Engel, B. Rafii, L. Blanchet, A. Suppers, E. Szymańska, J. J. Jansen, L. M. C. Buydens, 'Improving high-dimensional data fusion by exploiting the multivariate advantage', Chemometrics and Intelligent Laboratory Systems 156 (2016) 231–240. DOI: 10.1016/j.chemolab.2016.05.010, https://doi.org/10.1016/j.chemolab.2016.05.010
  2. T. Casian, B. Nagy, B. Kovács, D. L. Galata, E. Hirsch, A. Farkas, 'Challenges and Opportunities of Implementing Data Fusion in Process Analytical Technology—A Review', Molecules 27 (2022) 4846. DOI: 10.3390/molecules27154846, https://doi.org/10.3390/molecules27154846
  3. O. Rodionova, A. Pomerantsev, 'Multi-block DD-SIMCA as a high-level data fusion tool', Analytica Chimica Acta 1265 (2023) 341328. DOI: 10.1016/j.aca.2023.341328, https://doi.org/10.1016/j.aca.2023.341328
  4. P. P. Hanzelik, S. Gergely, J. Abonyi, A. Kummer, 'Data fusion of spectroscopic data for enhancing machine learning model performance', Digital Chemical Engineering 17 (2025) 100271. DOI: 10.1016/j.dche.2025.100271, https://doi.org/10.1016/j.dche.2025.100271
  5. E. Szymańska, 'Modern data science for analytical chemical data – A comprehensive review', Analytica Chimica Acta 1028 (2018) 1–10. DOI: 10.1016/j.aca.2018.05.038, https://doi.org/10.1016/j.aca.2018.05.038
  6. A. Smolinska, J. Engel, E. Szymanska, L. Buydens, L. Blanchet, 'General Framing of Low-, Mid-, and High-Level Data Fusion With Examples in the Life Sciences', in: M. Cocchi (ed.), Data Fusion Methodology and Applications (Data Handling in Science and Technology), Elsevier (2019), pp. 51–79. DOI: 10.1016/B978-0-444-63984-4.00003-X, https://doi.org/10.1016/B978-0-444-63984-4.00003-X
  7. S. Schwolow, N. Gerhardt, S. Rohn, P. Weller, 'Data fusion of GC-IMS data and FT-MIR spectra for the authentication of olive oils and honeys—is it worth to go the extra mile?', Analytical and Bioanalytical Chemistry 411 (2019) 6005–6019. DOI: 10.1007/s00216-019-01978-w, https://doi.org/10.1007/s00216-019-01978-w
  8. A. K. Smilde, T. Næs, K. H. Liland, 'Multiblock Data Fusion in Statistics and Machine Learning: Applications in the Natural and Life Sciences', Wiley (2019; online edition 2022). DOI: 10.1002/9781119600978, ISBN 9781119600961, https://doi.org/10.1002/9781119600978