DeepRNA-Twist: Language-Model-Guided RNA Torsion Angle Prediction with Attention-Inception Network

2025-05-04 Sun
RNA language model torsion and pseudo-torsion angles inception network deep learning RNA structure prediction bioinformatics long-range dependencies
1. Academic Background and Research MotivationWith the rapid development of life sciences and bioinformatics, research into RNA molecular structure and function has become a hot topic. RNA is not merely a carrier of genetic information, but also plays critical roles in regulation, catalysis, and various physiological processes. The three-dimensional structure of RNA directly affects its biological functions, and precise analysis of RNA structures is of great significance for basic science, drug design, and studies of disease mechanisms. However, the sequence-to-structure transformation for RNA is far more complex than for proteins—not only because the RNA backbone has seven main chain torsion angles (α, β, γ, δ, ε, ζ, χ), but also due to the involvement of complicated pseudo-torsion angles (η, θ), along with non-canonical base pairs, multiloops, triplexes, and other diverse structural factors—making high-precision RNA 3D structure prediction extremely challenging.
Traditional experimental methods for determining RNA structure, such as Nuclear Magnetic Resonance (NMR), X-ray crystallography, and cryo-Electron Microscopy (cryo-EM), are expensive and time-consuming. Earlier computational approaches, whether physics-based or statistical, struggled to adequately capture the highly complex and long-range dependencies between RNA sequence and structure. In recent years, deep learning has driven transformative advances in life sciences, with breakthroughs in protein secondary and tertiary structure prediction. Inspired by this, RNA structure prediction is also attempting to introduce more powerful machine learning methods, though progress is still limited.
In torsion angle prediction, mainstream methods such as spot-rna-1d have used Dilated Convolutional Neural Networks (Dilated CNN) to predict main-chain and pseudo-torsion angles from single-sequence inputs. Recently, rna-torsionbert took the lead in leveraging language model embeddings to predict RNA torsion angles and proposed a torsion angle-based structural quality assessment function, rna torsion-a. However, overall prediction accuracy remains improvable, particularly for highly variable torsion angles.
To address these challenges and needs, the authors designed deeprna-twist—a deep learning framework combining RNA language models, the novel attention-inception inside inception with dilated cnn module (2a3idc), and multi-head self-attention, aiming to efficiently and accurately predict RNA torsion and pseudo-torsion angles directly from sequence and achieve a breakthrough in RNA 3D structural analysis.
2. Source and Author InformationThe paper “deeprna-twist: language-model-guided rna torsion angle prediction with attention-inception network” is co-authored by Abrar Rahman Abir, Md Toki Tahmid, Rafiqul Islam Rayan, and M Saifur Rahman. All authors are affiliated with the Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology. The paper was published on March 23, 2025 by Oxford University Press in Briefings in Bioinformatics, Volume 26, Issue 3. As an open access article, it notably influences the advancement of bioinformatics analysis in RNA structure.
3. Detailed Explanation of Research Workflow1. Dataset Preparation and Feature RepresentationDataset PreparationThe study uses high-quality RNA structural data provided by spot-rna-1d as training and test sets. To further validate model generalizability and robustness, the authors also used the test set adopted by rna-torsionbert, which combines two authoritative datasets: RNA puzzles and CASP-RNA. The training set comprises 286 RNA chains (21,736 nucleotides), and the test set includes multiple datasets such as rnapuzzles (40 structures) and casp-rna (12 structures), with a total of 15,445 nucleotides and a maximum RNA length of 720 bases. The structural and length diversity of the dataset provides a solid foundation for the deep learning model to learn rich structural features.
Feature RepresentationThis study innovatively uses embeddings generated by the self-supervised RNA language model RINalmo (650M parameters) as input features. These embedding vectors (1280 dimensions) effectively capture the contextual semantics and structural correlations of RNA sequences, greatly enhancing the representational ability of single-sequence vectors. In this way, the model can learn long-term dependencies and deep structural signals, far outperforming the traditional one-hot encoding features.
2. deeprna-twist Deep Learning Framework DesignThe deeprna-twist architecture comprises three innovative modules:
(1) Transformer Encoder LayerFirst, the RINalmo embedding feature sequence of each nucleotide is processed by a Transformer Encoder layer. This module introduces a multi-head self-attention mechanism and a feed-forward network, combined with layer normalization and residual connections, to dynamically capture contextual dependencies among nucleotides in the sequence, effectively integrating both local and global structural information.
(2) 2a3idc Module (Attention Augmented Inception Inside Inception with Dilated CNN)This core module processes input in two parallel paths, each structured as follows: based on a four-branch Inception module, it captures multi-scale feature information with varying receptive fields (kernel sizes) and dilation rates, leveraging dilated convolution to efficiently model distant dependencies. After the dilated convolution, a multi-head attention mechanism is introduced in each path to further focus on critical signals at different positions in the sequence. Finally, the two outputs are concatenated and normalized, providing a rich and complementary high-dimensional representation for downstream prediction layers.
(3) Output Layer and Loss FunctionThe two-dimensional features are further processed by a 1D convolution and an additional attention module, then fed into a dense (fully connected) layer with 18 regression nodes, which respectively predict the sine and cosine values of 9 angles (7 backbone + 2 pseudo-torsion angles) for each nucleotide. The training objective is to minimize the mean squared error (MSE) between predicted and true values, ensuring effective optimization for periodicity and numerical continuity in the angles.
3. Training and Evaluation ProcedureTraining uses the Adam optimizer (learning rate 0.0001) for 120 epochs, with MSE as the loss function. In addition to the mainstream Mean Absolute Error (MAE), the evaluation also introduces MCQ (Mean Circular Quantities), which measures the overall angular similarity between predicted and experimental structures. MCQ better reflects the degree of structural reconstruction accuracy in 3D, making the assessment more specialized and practical.
4. Main Experimental Results and Logical Analysis1. General Performance ImprovementsExperiments show that deeprna-twist achieves state-of-the-art performance on authoritative test sets such as spot-rna-1d and rna-torsionbert, comprehensively outperforming leading existing methods like spot-rna-1d and rna-torsionbert. For example, on the spot-rna-1d test set, MAE for each angle declines by 10%-15% on average compared to previous methods. Angles with high variability such as α, ζ, and θ see particularly remarkable improvements, while low-variability angles such as δ, ε, and χ also retain optimal predictions. Broad stratification of sample lengths confirms that the model maintains stable accuracy even for long-chain RNAs, significantly improving its generalizability.
2. Prediction of Complex StructuresFor different nucleotide pairing types (e.g., unpaired, lone pairs, pseudoknots, triplets, non-canonical pairs, canonical nested pairs), the model achieves higher accuracy than spot-rna-1d even in complex tertiary structures (such as pseudoknots, multiloops, and non-canonical pairs), demonstrating that deeprna-twist effectively overcomes the challenging torsion angle prediction problems unsolved by previous methods for complex structures.
Further, compared with cutting-edge structure prediction algorithms (such as AlphaFold 3 and rhofold+) and mainstream torsion angle predictors, deeprna-twist obtains lower RMSD in independent structure reconstruction cases, displaying stronger 3D reconstruction capability.
3. Ablation and Comparative AnalysisAblation studies thoroughly validate the contribution of each innovative module. When one-hot encoding replaces RINalmo embeddings, MAE increases significantly, demonstrating the powerful structural information extraction capability of the language model. Removing the 2a3idc module results in the largest increase in error (average MAE rises by 3-5°), highlighting its crucial role in capturing multi-scale and multi-level long-range dependencies. Substituting or removing multi-head attention and the Transformer also obviously decreases accuracy. When compared with different RNA language models (such as birna-bert, rna-fm), RINalmo performs best, fully confirming the rationality and innovation of the model composition.
4. Structure Reconstruction Case StudiesTaking PDB structures (4r4v and 7ptk) as examples, the authors reconstructed atomic structures using deeprna-twist-predicted torsion angles. The predicted-to-experimental structure RMSDs were 3.31 Å and 6.59 Å, respectively—significantly better than similar algorithms—demonstrating the model’s practical application potential for downstream 3D reconstruction and functional analysis.
5. Conclusion and Value AnalysisAs a new-generation bioinformatics tool for RNA structure, deeprna-twist achieves both theoretical and practical breakthroughs for the RNA torsion angle prediction problem. Its scientific value is reflected mainly in:
Scientific Advancement: For the first time, it achieves high-precision torsion angle prediction based on large-scale RNA language models combined with attention mechanisms, inception-dilated convolution, and more, laying the groundwork for large-scale analysis of more complex RNA 3D structures.
Engineering Practicality: The model requires only sequence for high-quality structure prediction, without needing composite input or templates, greatly accelerating downstream processes such as drug screening, structure alignment, and functional annotation.
Model Innovation: The proposed 2a3idc module combines scale-aware, multi-head attention, and long-range dependency capture, improving model expressivity while reducing parameter count—an important paradigm for structural bioinformatics algorithm design.
Wide Applicability: The model is open sourced (https://github.com/abrarrahmanabir/deeprna-twist), providing global researchers with an easy-to-use, efficient, and transferable toolkit that has the potential to become a standard predictive solution in RNA research.
6. Research Highlights and Prospectsdeeprna-twist innovatively integrates large-model embeddings, improved inception modules, dilated convolutions, and multi-head attention. Multiple ablation and comparative experiments rigorously prove its significant advantages.
In predicting highly variable torsion angles and complex tertiary structure regions—which are hugely challenging—it demonstrates extremely high accuracy and stability, overcoming several long-standing difficulties in RNA structure prediction.
Its unique evaluation system (including the MCQ metric) more scientifically reflects the spatial similarity between predicted and real structures, promoting evolution in predictive methodology.
Despite these advances, because of the extreme flexibility and biophysical complexity of RNA structures, achieving ultra-low error rates still awaits deeper integration of larger data, interdisciplinary knowledge, and biophysical constraints. In response, the author team plans to further optimize feature extraction and network structure, bringing more innovative achievements to RNA structural research.
7. References and AcknowledgmentsThis work is supported by the Basic Research Fund of Bangladesh University of Engineering and Technology. Related code, data, and detailed references can be found in the original paper and authors’ GitHub homepage.
This report provides an in-depth interpretation of the design philosophy, research workflow, core technologies, major results, and application prospects of the innovative deeprna-twist algorithm, not only injecting new vitality into RNA structure prediction, but also promoting boundary-breaking innovation at the intersection of bioinformatics and artificial intelligence.