


Open source! Hong Kong Chinese, MIT, and Fudan propose the first RNA cornerstone model
Unlike the protein field, research in the RNA field often lacks sufficient annotation data. For example, 3D data only has more than 1,000 RNAs. This greatly limits the development of machine learning methods in RNA structure-function prediction tasks.
In order to make up for the lack of annotated data, this article demonstrates a cornerstone model that can provide rich structural and functional knowledge for various RNA studies - RNA foundation model ( RNA-FM). As the world's first RNA cornerstone model trained in an unsupervised manner based on 23 million unlabeled RNA sequences, RNA-FM mines the evolutionary and structural patterns contained in RNA sequences.
It is worth noting that RNA-FM only needs to match a simple downstream model or only provide embedding, and it can achieve performance far exceeding SOTA in many downstream tasks, such as It can be improved by 20% in secondary structure prediction and 30% in distance map prediction. Large-scale experiments have proven that the model is highly generalizable and can even be used for COVID-19 and regulatory fragments of mRNA.
- ##Preprint of the paper: https://arxiv. org/abs/2204.00300
- Code and model: https://github.com/ml4bio/RNA-FM
- Server: https://proj.cse.cuhk.edu.hk/rnafm
In recent years, biological computing methods based on deep learning have made breakthrough progress in the field of proteins. The most famous milestone is the end-to-end protein 3D structure prediction framework AlphaFold2 developed by the Google DeepMind team. However, protein is only one type of many biological molecules. Gene (DNA/RNA), as the source of protein production, contains more basic information than the latter and has more important research value.
Generally speaking, proteins are the products of translation from RNA used for coding, that is, mRNA. A fixed mRNA can be translated into a fixed protein sequence. In fact, this part of coding RNA only accounts for 2% of all RNA sequences, and the remaining 98% is non-coding RNA (ncRNA). Although ncRNAs are not directly "translated" into proteins, they fold into tertiary structures with specific functions and play a regulatory role in the translation process of mRNA or other biological functions. Therefore, analyzing the structure and function of ncRNA is a more basic and complex research than protein analysis.
However, compared to the protein field, where computational methods are more mature, RNA-based structure and function prediction is still in its early stages, and computational methods originally applicable to the protein field are difficult to directly migrate. to the RNA field. The main limitation of these computational methods is that annotation of RNA data is usually difficult to obtain, and it requires a lot of experimental resources and time to complete the annotation of a small amount of data. Most computational methods require a large amount of annotated data for supervision to achieve high performance. Although there is not much annotated data, the RNA field has actually accumulated a lot of unannotated sequence data. The method of this article is to use these unlabeled data to provide additional effective information for various downstream tasks.
Based on this consideration, Hong Kong Chinese, MIT, Fudan and Shanghai Artificial Intelligence Laboratory teams proposed an unsupervised method to The RNA foundation model (RNA-FM) is trained on 23 million label-free pure RNA sequences. Although the data does not provide annotation information during the training process, RNA-FM still mines the evolutionary and structural patterns contained in these RNA sequences in an unsupervised manner.
If RNA-FM can be effectively applied to downstream RNA structure and function prediction tasks, these computational methods will surely benefit from the knowledge induced by RNA-FM and achieve better performance Performance improvements. The upstream pre-training and downstream migration and application framework of RNA-FM are shown in the figure below.
In order to confirm whether the pre-trained RNA-FM has learned "knowledge" from a large amount of unlabeled data and What kind of "knowledge" has been learned? The articleconducts a series of analyzes on embedding.
First, a simple clustering comparison of various features was conducted directly through UMAP, and it was found that the embedding from pre-trained RNA-FM was better than other Embedding forms clusters with more distinct RNA species. This means that the embedding of RNA-FM does contain structural or functional information for distinguishing RNA species.
Then, the article also uses trajectory inference (Trajectory inference) to predict the evolution of lncRNA from different species through RNA-FM embedding. From the streamplot below, the predicted pseudo-time of evolution between species is roughly consistent with the real species evolution information, indicating that RNA-FM embedding also contains part of the evolutionary information.
It is worth noting that, whether it is community information of RNA species or evolutionary information of lncRNA, RNA-FM has not been directly exposed to these labels during training. RNA-FM discovers patterns related to structure, function and evolution from pure sequences in a completely self-supervised manner.
More experimental results
In addition to directly analyzing the embedding of RNA-FM , the article also attempts to introduce RNA-FM to various downstream RNA structure prediction tasks, including secondary structure, contact prediction, distance prediction, and tertiary structure prediction, and has achieved obvious results. promote.
Especially in terms of secondary structure prediction, the article uses RNA-FM as the backbone and only uses a simple ResNet network as the downstream model, surpassing two public data sets. The other 12 state-of-the-art methods are superior to the best UFold by 3-5 percentage points in F1score. In the head-to-head comparison with UFold, RNA-FM performs better in most RNA categories. More than UFold. If RNA-FM is combined with E2Efold, a further 5% performance improvement can be achieved.
Using RNA-FM to conduct a complete analysis of COVID-19 data, including using RNA-FM to accurately predict key regulatory elements in the COVID-19 reference genome (29870 nt), and using RNA-FM embedding to roughly predict the evolutionary trends of major COVID-19 variants.
Therefore, the article
further attempts to introduce RNA-FM into downstream RNA function prediction tasks, such as using RNA-FM embedding. Prediction of RNA-protein roles.
Experiments have proven that the introduction of RNA-FM embedding improves the performance of the model, and in some cases even achieves prediction results that match real secondary structure information as input.
finally attempts to use RNA -FM performs functional prediction of protein expression based on 5'UTR on mRNA. Although mRNA does not belong to ncRNA, the 5'UTR on it is a region that is not translated but has regulatory functions, which is consistent with the characteristics of ncRNA and does not appear in the training data.
As you can see from the figure below, models that include RNA-FM embedding are always better than models that do not. Although the performance improvement is relatively limited, it partly shows that RNA-FM also has certain generalization ability on non-ncRNA data.
Conclusion
In general, this article uses unlabeled RNA sequence data to pre-train the language model RNA-FM, and through direct or indirect methods, a series of structural or functional Comprehensive verification on different tasks proves that RNA-FM can indeed effectively improve the performance of computing methods in downstream tasks.
The emergence of RNA-FM has alleviated the current situation of RNA labeled data to a certain extent, and provides other researchers with a convenient interface to access large quantities of unlabeled data, which will As a basic model in the RNA field, it provides strong support and help for various research in this field.
About the author
This article has two co-first authors. Chen Jiayang is a research assistant at the Chinese University of Hong Kong. Hu Zhihang is a doctoral candidate at the Chinese University of Hong Kong.
This article has two corresponding authors. Sun Siqi, young researcher at Fudan University Intelligent Complex Systems Laboratory and Shanghai Artificial Intelligence Laboratory, homepage https://intersun.github.io.
Li Yu, Assistant Professor at the Chinese University of Hong Kong, Visiting Assistant Professor at MIT James Collins Lab, Research Scientist at Broad Institute of MIT and Harvard, Visiting Scholar at Wyss Institute at Harvard University, Forbes 30 Under 30 Asia list–Class of 2022, Healthcare & Science. Home page: https://liyu95.com.
The above is the detailed content of Open source! Hong Kong Chinese, MIT, and Fudan propose the first RNA cornerstone model. For more information, please follow other related articles on the PHP Chinese website!

The unchecked internal deployment of advanced AI systems poses significant risks, according to a new report from Apollo Research. This lack of oversight, prevalent among major AI firms, allows for potential catastrophic outcomes, ranging from uncont

Traditional lie detectors are outdated. Relying on the pointer connected by the wristband, a lie detector that prints out the subject's vital signs and physical reactions is not accurate in identifying lies. This is why lie detection results are not usually adopted by the court, although it has led to many innocent people being jailed. In contrast, artificial intelligence is a powerful data engine, and its working principle is to observe all aspects. This means that scientists can apply artificial intelligence to applications seeking truth through a variety of ways. One approach is to analyze the vital sign responses of the person being interrogated like a lie detector, but with a more detailed and precise comparative analysis. Another approach is to use linguistic markup to analyze what people actually say and use logic and reasoning. As the saying goes, one lie breeds another lie, and eventually

The aerospace industry, a pioneer of innovation, is leveraging AI to tackle its most intricate challenges. Modern aviation's increasing complexity necessitates AI's automation and real-time intelligence capabilities for enhanced safety, reduced oper

The rapid development of robotics has brought us a fascinating case study. The N2 robot from Noetix weighs over 40 pounds and is 3 feet tall and is said to be able to backflip. Unitree's G1 robot weighs about twice the size of the N2 and is about 4 feet tall. There are also many smaller humanoid robots participating in the competition, and there is even a robot that is driven forward by a fan. Data interpretation The half marathon attracted more than 12,000 spectators, but only 21 humanoid robots participated. Although the government pointed out that the participating robots conducted "intensive training" before the competition, not all robots completed the entire competition. Champion - Tiangong Ult developed by Beijing Humanoid Robot Innovation Center

Artificial intelligence, in its current form, isn't truly intelligent; it's adept at mimicking and refining existing data. We're not creating artificial intelligence, but rather artificial inference—machines that process information, while humans su

A report found that an updated interface was hidden in the code for Google Photos Android version 7.26, and each time you view a photo, a row of newly detected face thumbnails are displayed at the bottom of the screen. The new facial thumbnails are missing name tags, so I suspect you need to click on them individually to see more information about each detected person. For now, this feature provides no information other than those people that Google Photos has found in your images. This feature is not available yet, so we don't know how Google will use it accurately. Google can use thumbnails to speed up finding more photos of selected people, or may be used for other purposes, such as selecting the individual to edit. Let's wait and see. As for now

Reinforcement finetuning has shaken up AI development by teaching models to adjust based on human feedback. It blends supervised learning foundations with reward-based updates to make them safer, more accurate, and genuinely help

Scientists have extensively studied human and simpler neural networks (like those in C. elegans) to understand their functionality. However, a crucial question arises: how do we adapt our own neural networks to work effectively alongside novel AI s


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

WebStorm Mac version
Useful JavaScript development tools

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

SublimeText3 Chinese version
Chinese version, very easy to use

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool
