Liu, Barr, and Reinitz (2020) showed that a thermodynamic model of enhancer-driven transcription in the early Drosophila embryo can be re-expressed as a deep neural network in which every layer has a chemical interpretation, trained by stochastic gradient descent with about a hundred-fold algorithmic speedup over simulated annealing. The demonstration, however, relied on extensive prior knowledge of the regulatory roles of the transcription factors involved and was validated on only a few hundred measurements from one Drosophila system. The open problem is to generalize this framework so that it works without such prior knowledge: inferring the functional roles of transcription factors (activator, quencher, or coactivator) directly from data without exhaustive combinatorial enumeration, and training and validating the model on genomic-scale paired datasets in organisms beyond Drosophila.

Background

In multicellular organisms, the decision of when and where a gene is transcribed is controlled largely by regulatory DNA (enhancers) read by proteins called transcription factors (TFs), which bind specific short DNA sequence patterns. A standard representation of a TF's binding preference is a position weight matrix (PWM), an additive model assigning a binding-energy contribution to each nucleotide at each position of the binding site. In metazoan animals, different cell types carry the same DNA but differ in the concentrations of the TFs they express, so a quantitative model of transcriptional control must predict gene expression from both the regulatory DNA sequence and the TF concentrations present.

Thermodynamic models of transcription (also called biophysical or occupancy models) provide such a framework: from the PWMs and TF concentrations they compute the probability that each TF occupies its binding sites using equilibrium statistical mechanics, then apply phenomenological regulatory rules — activators increase transcription, quenchers (short-range repressors) suppress nearby activators, and coactivators act cooperatively — to predict the transcription rate. Kim et al. (2013) built such a model for enhancers of the even-skipped gene in the blastoderm embryo of the fruit fly Drosophila, an early embryonic stage where the network of 'gap gene' TFs patterns the body axis; fitting it required roughly ten thousand lines of C++ and simulated annealing, a stochastic global-optimization method needing millions of loss-function evaluations.

Liu, Barr, and Reinitz (2020) observed that this thermodynamic model is itself a deep neural network (DNN) — its computation can be laid out as stacked layers, each with an exact chemical interpretation (PWM convolution to binding affinities, occupancy computation, coactivation, quenching, summation and thresholding to a synthesis rate). They reimplemented it in Keras/TensorFlow with custom layers (about 600 lines of Keras code, 52 unknown parameters) and trained it by backpropagation and stochastic gradient descent (SGD), obtaining roughly a hundred-fold algorithmic speedup: about ten thousand loss evaluations versus several million. The model was trained and biologically validated only on a small Drosophila dataset (four fusion constructs, 232 unique observations).

Two dependencies blocked generalization in the authors' assessment. First, the model required prior knowledge of each TF's functional role (which TFs are activators, quenchers, or coactivators). Without that knowledge, a closely related thermodynamic model applied to mouse erythropoiesis (blood-cell development) by Bertolino et al. (2016), analyzed further by Repele et al. (2019), had to consider all possible combinations of TF functional roles and select the best-fitting assignment — computationally prohibitive as the number of TFs grows. The authors suggested reinforcement learning (a machine-learning approach in which an agent learns a discrete decision policy through reward feedback) as a possible efficient alternative to this exhaustive combinatorial search. Second, although genomic-scale datasets pairing sequence, TF concentrations, and expression measurements (for example STARR-seq, a type of massively parallel reporter assay, or MPRA, that measures enhancer activity for thousands of sequences at once) have become available in flies and mammals, the framework has not been demonstrated at that scale, and a mismatch remained between the algorithmic speedup and the achieved wall-clock speedup, which the authors attributed to room for improvement in the TensorFlow implementation for this unconventional architecture.

Problem Statement

The problem, as posed by the Discussion of Liu, Barr, and Reinitz (2020), is to remove the two dependencies that keep their fully interpretable thermodynamic deep neural network (DNN) of transcriptional control a testbed-scale demonstration, with one common objective: learn the regulatory code — predicting quantitative gene expression from regulatory DNA sequence together with transcription factor (TF) concentrations, with every network layer retaining its chemical interpretation — at genomic scale, in an organism other than Drosophila, without supplying the model the functional roles of the TFs in advance. Concretely, an answer must jointly deliver: (1) a computationally tractable method, in the spirit of the suggested reinforcement-learning approach (an agent learning a discrete decision policy through reward feedback) but not restricted to it, that infers each TF's functional role (activator, quencher, coactivator, or a learned generalization that is identifiable from the fitted model and maps to these classes) from training data, replacing the exhaustive enumeration over all role combinations that made the mouse Cebpa studies (Bertolino et al. 2016; Repele et al. 2019) computationally prohibitive; and (2) a trained, layer-wise interpretable model of this class validated on a genomic-scale dataset pairing sequence, TF concentrations, and expression measurements (for example STARR-seq or related massively parallel reporter data with quantified TF levels) in an organism beyond Drosophila, achieving predictive performance with inferred roles consistent with independent biological evidence. The relative terms 'genomic scale' and 'beyond Drosophila' are to be judged as established in the field — datasets orders of magnitude larger than the 232-observation Drosophila test set, and a non-Drosophilid organism — rather than by an invented numerical threshold. Likewise, the empirical criteria of deliverable (2) — achieving predictive performance while inferring roles consistent with independent biological evidence — are to be judged against what is reported for interpretable sequence-to-expression models on comparable data, rather than by an invented numerical threshold.

References

  1. Yi Liu, Kenneth Barr, John Reinitz (2020), 'Fully interpretable deep learning model of transcriptional control', Bioinformatics 36(Supplement_1):i499–i507, doi:10.1093/bioinformatics/btaa506, https://doi.org/10.1093/bioinformatics/btaa506
  2. Yi Liu, Kenneth Barr, John Reinitz (2019), 'Fully Interpretable Deep Learning Model of Transcriptional Control', bioRxiv preprint, doi:10.1101/655639, https://doi.org/10.1101/655639
  3. Ah-Ram Kim, Carlos Martinez, John Ionides, Alexandre F. Ramos, Michael Z. Ludwig, Nobuo Ogawa, David H. Sharp, John Reinitz (2013), 'Rearrangements of 2.5 Kilobases of Noncoding DNA from the Drosophila even-skipped Locus Define Predictive Rules of Genomic cis-Regulatory Logic', PLoS Genetics 9(2):e1003243, doi:10.1371/journal.pgen.1003243, https://doi.org/10.1371/journal.pgen.1003243
  4. Eric Bertolino, John Reinitz, Manu (2016), 'The analysis of novel distal Cebpa enhancers and silencers using a transcriptional model reveals the complex regulatory logic of hematopoietic lineage specification', Developmental Biology 413(1):128–144, doi:10.1016/j.ydbio.2016.02.030, https://doi.org/10.1016/j.ydbio.2016.02.030
  5. Andrea Repele, Shawn Krueger, Tapas Bhattacharyya, Michelle Y. Tuineau, Manu (2019), 'The regulatory control of Cebpa enhancers and silencers in the myeloid and red-blood cell lineages', PLOS ONE 14(6):e0217580, doi:10.1371/journal.pone.0217580, https://doi.org/10.1371/journal.pone.0217580
  6. Alan U. Sabino, Drielly M. Guerreiro, Ah-Ram Kim, Alexandre F. Ramos, John Reinitz (2025), 'Characterizing the regulatory logic of transcriptional control at the DNA sequence level by ensembles of thermodynamic models', Bioinformatics 41(10):btaf534, doi:10.1093/bioinformatics/btaf534, https://doi.org/10.1093/bioinformatics/btaf534
  7. Payam Dibaeinia, Saurabh Sinha (2021), 'Deciphering enhancer sequence using thermodynamics-based models and convolutional neural networks', Nucleic Acids Research 49(18):10309–10327, doi:10.1093/nar/gkab765, https://doi.org/10.1093/nar/gkab765
  8. Ali Tuğrul Balcı, Mark Maher Ebeid, Panayiotis V. Benos, Dennis Kostka, Maria Chikina (2023), 'An intrinsically interpretable neural network architecture for sequence-to-function learning', Bioinformatics 39(Supplement_1):i413–i422, doi:10.1093/bioinformatics/btad271, https://doi.org/10.1093/bioinformatics/btad271
  9. Amin Safaeesirat, Hoda Taeb, Eldon Emberly (2026), 'Inference of enhancer-specific transcription factor interactions from gene expression data using a biophysical model', bioRxiv preprint (posted 2026-06-08), doi:10.64898/2026.06.03.729923, https://doi.org/10.64898/2026.06.03.729923
  10. Michael J. Gaiewski, Robert A. Drewell, Jacqueline M. Dresch (2021), 'Fitting thermodynamic-based models: Incorporating parameter sensitivity improves the performance of an evolutionary algorithm', Mathematical Biosciences 342:108716, doi:10.1016/j.mbs.2021.108716, https://doi.org/10.1016/j.mbs.2021.108716