Making QSAR models activity-cliff-aware: pair-based training, representations beyond fingerprints, generalisable error–cliff coupling, and prospective validation of activity-cliff prediction

Quantitative structure-activity relationship (QSAR) models systematically fail on activity cliffs (ACs) — pairs of near-identical molecules with large potency differences. Dablander, Hanser, Lambiotte and Morris established this failure mode quantitatively (Journal of Cheminformatics 15:47, 2023) and observed an approximately linear coupling between molecule-level prediction error and pair-level cliff-detection performance — an observation they flag as resting on only nine models. Their published conclusions identify one open direction: that 'it might be possible to considerably boost the performance of common QSAR models by developing techniques to increase their AC-sensitivity', with self-supervised GNN pretraining named as one potential angle towards definitively beating classical fingerprint featurisations. The preprint version (arXiv:2301.13644) additionally proposes a concrete instance of that direction — an AC-sensitive twin-network (Siamese-style) pair-based training scheme for deep-learning models — which the authors explicitly leave to future evaluation. The subsequent literature shows vigorous method development (triplet-loss, contrastive, weighted-graph and pairwise-delta models) but no leakage-controlled demonstration of reliable cliff prediction, no replication or refutation of the error–cliff coupling, and no prospective experimental validation: the source's open direction remains unresolved.

Background

In drug discovery, a quantitative structure-activity relationship (QSAR) model is a regression function $f$ trained to map a molecular structure to a potency label, typically a negative logarithm of an equilibrium constant or IC50 value. Molecular structure is encoded by a representation: extended-connectivity fingerprints (ECFPs) are fixed binary vectors counting the presence of circular atom neighbourhoods up to a chosen radius; physicochemical-descriptor vectors (PDVs) collect precomputed properties such as logP and molecular weight; graph isomorphism networks (GINs) are graph neural networks (GNNs) that learn differentiable molecular features directly from the molecular graph by message passing. Model quality is usually reported as mean absolute error (MAE) over a held-out test set of individual molecules.

An activity cliff (AC) is a pair of structurally similar compounds whose potencies differ drastically — an extreme discontinuity of the structure-activity relationship (SAR). ACs are usually operationalised on matched molecular pairs (MMPs): two compounds that differ only by a small structural transformation at a single site. Fixing a critical potency-difference threshold $d_{\text{crit}}$ (Dablander et al. use $d_{\text{crit}}=1.5$ log-units), an MMP is labelled an AC if the potency difference exceeds $d_{\text{crit}}$. Any QSAR regressor can be repurposed as an AC classifier by comparing its predictions for the two compounds of a pair, and its pair-level detection quality is measured by AC-sensitivity (the fraction of true cliffs detected) and the Matthews correlation coefficient (MCC), a balanced classification metric suited to imbalanced problems, since cliffs are rare among all similar pairs.

The motivating study (Dablander et al., J. Cheminformatics 15:47, 2023; preprint arXiv:2301.13644) built nine QSAR models (three representations — ECFP, PDV, GIN — crossed with three regressors — random forest, k-nearest neighbours, multilayer perceptron) for three targets (dopamine receptor D2, factor Xa, SARS-CoV-2 main protease), using a pair-level data-splitting scheme that distinguishes pairs where one compound's activity is known during training from pairs where both activities are unknown. Its findings: AC-sensitivity is universally low when both activities are unknown, strongly supporting the hypothesis that QSAR methods regularly fail on cliffs; GINs are competitive or superior for AC classification while task-agnostic ECFPs still win at molecule-level QSAR; and across the nine models the QSAR-MAE and AC-MCC are approximately linearly related — a coupling the authors themselves flag as resting on only nine models. The preprint version additionally proposes a concrete remedy: twin-network (Siamese-style) pair-based training, in which a regressor $f$ is trained on compound pairs $\{s,\tilde{s}\}$ with the loss

$$w_{\{s,\tilde{s}\}}\left[(a(s)-f(s))^2+(a(\tilde{s})-f(\tilde{s}))^2+w_{\text{diff}}\left((a(s)-a(\tilde{s}))-(f(s)-f(\tilde{s}))\right)^2\right],$$

possibly in two phases (general compound pairs, then matched molecular pairs weighted by activity difference), with the intent to evaluate it in future work.

These results sit inside a broader debate catalysed by the MoleculeACE benchmark (a public suite of bioactivity datasets with similarity-aware evaluation splits) of van Tilborg, Alenicheva and Grisoni (J. Chem. Inf. Model. 62:5938–5951, 2022), which showed that on similarity-aware splits dominated by cliffs, classical fingerprints with simple learners remain hard to beat for deep learning. A further methodological hazard, established by later leakage-controlled studies, is data leakage: because both compounds of a pair derive from the same chemical neighbourhood, pair-level splits can place near-duplicates of test compounds in the training set and inflate apparent cliff-prediction accuracy. Evaluations of cliff prediction therefore need to control for compound-level leakage, i.e. prevent near-duplicate compounds from appearing on both sides of the train/test boundary; this can be pursued through different designs, among them scaffold- or molecular-component-grouped splits, explicit near-duplicate filtering, or the pair-level MMP scheme of the source study.

Problem Statement

The overarching objective is to make molecular machine learning genuinely activity-cliff-aware, and to establish with rigorous, leakage-controlled evidence when such models detect cliffs. The source's own open direction — developing techniques to increase the AC-sensitivity of QSAR models — is carried by strands (1) and (2); strands (3) and (4) arise as extensions of the study's acknowledged limitations and of the subsequent literature, and bear on the same objective.

(1) Training schemes. Starting from standard QSAR regressors (e.g. ECFP-MLP, GIN-MLP), can pair-based training schemes increase AC-sensitivity? How does the balance between AC-sensitivity and molecule-level QSAR accuracy (MAE) depend on the pair weight $w_{\{s,\tilde{s}\}}$ and the difference weight $w_{\text{diff}}$, and can AC-sensitivity be increased without a compensating loss in MAE? (The preprint anticipated that its twin-network scheme would increase AC-sensitivity 'and thus overall QSAR performance'; whether sensitivity gains in fact carry an accuracy cost is the open empirical question.)

(2) Representations. Can representation learning reliably surpass fingerprint-based baselines such as ECFP-MLP for molecule-level QSAR, while retaining or improving pair-level cliff detection in novel chemical regions (that is, under distribution-shifted evaluation such as out-of-distribution or scaffold-disjoint test sets), consistently across targets rather than in single-dataset wins?

(3) Generality of the error-cliff coupling (extension). The source reports the coupling as an observation that 'only rests on nine models' yet is 'highly consistent across MMP-sets and pharmacological targets'; it does not pose its generalisation as future work. Does the empirical coupling between molecule-level QSAR error and pair-level cliff-detection performance — the approximately linear relationship between QSAR-MAE and AC-MCC observed over nine models on three targets — persist across other pharmacological targets, different activity units and assay-noise regimes, larger and more diverse chemical libraries, and different cliff-definition thresholds $d_{\text{crit}}$?

(4) Practical validation (extension). Do AC-prediction baselines and cliff-aware trained models actually improve prospective compound optimisation and experimental decision-making, including how to balance cliff-detection sensitivity against precision under real experimental cost constraints? This strand presupposes new synthesis and assay work and is therefore downstream of the modelling question posed by the source.

Progress on the source's open direction is evidenced by leakage-controlled studies of strands (1) and (2): whether cliff-sensitivity can be increased without a compensating loss in molecule-level QSAR accuracy, and how that trade-off depends on the training scheme and on the pair and difference weights; and whether any representation approach reliably beats fingerprint baselines on both molecule-level and pair-level tasks in novel chemical regions, consistently across targets rather than in single-dataset wins. These strands are refutable in either direction: a leakage-controlled demonstration that none of the examined schemes raises AC-sensitivity without a compensating MAE loss, or that no representation approach consistently beats ECFP-style baselines, is a scientific outcome of the enquiry.

References

1. Markus Dablander, Thierry Hanser, Renaud Lambiotte, Garrett M. Morris. Exploring QSAR models for activity-cliff prediction. Journal of Cheminformatics 15:47, 2023. DOI: 10.1186/s13321-023-00708-w. https://doi.org/10.1186/s13321-023-00708-w. Preprint version: arXiv:2301.13644, https://arxiv.org/abs/2301.13644 (the twin-network future-research section appears in the preprint version but was removed from the published version of record)
2. Markus F. Dablander. Investigating graph neural networks and classical feature-extraction techniques in activity-cliff and molecular property prediction. DPhil thesis, University of Oxford, 2023. DOI: 10.5287/ora-xkardwd6z. https://ora.ox.ac.uk/objects/uuid:07029718-1442-45c2-a589-d929871318fe
3. Sam Money-Kyrle, Markus Dablander, Thierry Hanser, Stephane Werner, Charlotte M. Deane, Garrett M. Morris. On Improving Graph Neural Networks for QSAR by Pre-training on Extended-Connectivity Fingerprints. arXiv:2605.10722, 2026. https://arxiv.org/abs/2605.10722
4. Derek van Tilborg, Alisa Alenicheva, Francesca Grisoni. Exposing the Limitations of Molecular Machine Learning with Activity Cliffs. Journal of Chemical Information and Modeling 62(24):5938-5951, 2022. DOI: 10.1021/acs.jcim.2c01073. https://doi.org/10.1021/acs.jcim.2c01073
5. Xinxin Yu, Yimeng Wang, Long Chen, Weihua Li, Yun Tang, Guixia Liu. ACtriplet: An improved deep learning model for activity cliffs prediction by integrating triplet loss and pre-training. Journal of Pharmaceutical Analysis 15:101317, 2025. DOI: 10.1016/j.jpha.2025.101317. https://doi.org/10.1016/j.jpha.2025.101317
6. Xiuyuan Hu, Guoqing Liu, Yang Zhao, Hao Zhang. Activity cliff-aware reinforcement learning for de novo drug design. Journal of Cheminformatics 17:54, 2025. DOI: 10.1186/s13321-025-01006-3. https://doi.org/10.1186/s13321-025-01006-3
7. Yiwei Chen, Tingfang Wu, Yelu Jiang, Liangpeng Nie, Geng Li, Yi Zhang, Zhenglong Zhou, Jia Xu, Lijun Quan, Qiang Lyu. MAPCliff-WMGR: Exploring Activity Cliffs in Molecular Activity Prediction Enhanced by Weighted Molecular Graph Representations. Journal of Chemical Information and Modeling 65:12978-12989, 2025. DOI: 10.1021/acs.jcim.5c01915. https://doi.org/10.1021/acs.jcim.5c01915
8. Yueming Yin, Haifeng Hu, Jitao Yang, Chun Ye, Wilson Wen Bin Goh, Adams Wai-Kin Kong, Jiansheng Wu. OLB-AC: toward optimizing ligand bioactivities through deep graph learning and activity cliffs. Bioinformatics 40(6):btae365, 2024. DOI: 10.1093/bioinformatics/btae365. https://doi.org/10.1093/bioinformatics/btae365
9. Samuel S. Agboola, Oluwaseun E. Agboola, Oluwaseun Ruth Olasehinde, Temitope C. Aribigbola, Anuoluwapo Bukolo Shaleye, I. Adefolake Alebiosu, Zainab A. Ayinla, Babatunji E. Oyinloye, Olutosin S. Ilesanmi, Idowu O. Omotuyi, Abel Kolawole Oyebamiji. Activity cliffs resist prediction within and across protein kinases: a leakage-controlled machine-learning analysis of structure-potency discontinuities. Bioorganic Chemistry 181:110406, 2026. DOI: 10.1016/j.bioorg.2026.110406. https://doi.org/10.1016/j.bioorg.2026.110406
10. Ryuto Abe, Tomoyuki Miyao, Jürgen Bajorath. Accurate prediction of activity cliff compounds based on bioactivity profiles depends on assay nearest neighbor relationships. Journal of Cheminformatics 18:72, 2026. DOI: 10.1186/s13321-026-01210-9. https://doi.org/10.1186/s13321-026-01210-9
11. Tiago Janela, Jürgen Bajorath. Anatomy of Potency Predictions Focusing on Structural Analogues with Increasing Potency Differences Including Activity Cliffs. Journal of Chemical Information and Modeling 63:7032-7044, 2023. DOI: 10.1021/acs.jcim.3c01530. https://doi.org/10.1021/acs.jcim.3c01530
12. Zachary Fralish, Ashley Chen, Paul Skaluba, Daniel Reker. DeepDelta: predicting ADMET improvements of molecular derivatives with deep learning. Journal of Cheminformatics 15:101, 2023. DOI: 10.1186/s13321-023-00769-x. https://doi.org/10.1186/s13321-023-00769-x
13. Zachary Fralish, Paul Skaluba, Daniel Reker. Leveraging bounded datapoints to classify molecular potency improvements. RSC Medicinal Chemistry 15:2474-2482, 2024. DOI: 10.1039/d4md00325j. https://doi.org/10.1039/d4md00325j
14. Pawel Dabrowski-Tumanski, Bartosz Topolski, Dariusz Plewczynski, Tomasz Jetka. The Geometry of Activity Cliffs: Representation Dependence and Multi-Scale Characterization of Activity Landscapes. arXiv:2605.30831, 2026. https://arxiv.org/abs/2605.30831
15. Zijing Liu, Xinni Zhang, Yankai Chen, Bin Feng, Mingjun Yang, Zenglin Xu, Yu Li, Philip S. Yu, Irwin King. Incorporating 3D Structural Information for Activity Cliff Analysis. Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, pages 11552-11563, 2026. DOI: 10.1145/3770855.3819046. https://doi.org/10.1145/3770855.3819046
16. Krishna Harish. Local structure-activity landscape roughness detects the activity-cliff failure mode that standard QSAR reliability methods miss. Research Square preprint, 2026. DOI: 10.21203/rs.3.rs-10071162/v1. https://doi.org/10.21203/rs.3.rs-10071162/v1
17. Zhixiang Cheng, Hongxin Xiang, Pengsen Ma, Li Zeng, Xin Jin, Xixi Yang, Jianxin Lin, Xinxin Feng, Yang Deng, Changhui Deng, Bosheng Song, Xiangxiang Zeng. MaskMol: knowledge-guided molecular image pre-training framework for activity cliffs with pixel masking. BMC Biology 23:279, 2025. DOI: 10.1186/s12915-025-02389-3. https://doi.org/10.1186/s12915-025-02389-3
18. Kenneth López-Pérez, Ramón Alain Miranda-Quintana. Extended Activity Cliffs-Driven Approaches on Data Splitting for the Study of Bioactivity Machine Learning Predictions. Molecular Informatics 44:e202400054, 2024. DOI: 10.1002/minf.202400054. https://doi.org/10.1002/minf.202400054
19. Markus Dablander, Thierry Hanser, Renaud Lambiotte, Garrett M. Morris. Sort & Slice: a simple and superior alternative to hash-based folding for extended-connectivity fingerprints. Journal of Cheminformatics 16:135, 2024. DOI: 10.1186/s13321-024-00932-y. https://doi.org/10.1186/s13321-024-00932-y