search
HomeTechnology peripheralsAINVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

Author | Liu Shengchao

Editor | Kaixia

Starting in 2021, the combination of big language and multi-modality has swept the machine learning research community .

With the development of large models and multi-modal applications, can we apply these techniques to drug discovery? And, can these natural language textual descriptions bring new perspectives to this challenging problem? The answer is yes, and we are optimistic about it

Recently, Canada’s Montreal Institute for Learning Algorithms (Mila), NVIDIA Research, University of Illinois at Urbana-Champaign (UIUC), Princeton University and California Institute of Technology The college's research team jointly learns the chemical structure and text description of molecules through comparative learning strategies, and proposes a multi-modal molecular structure-text model MoleculeSTM.

The research is titled "Multi-modal molecule structure–text model for text-based retrieval and editing" and was published in "Nature Machine Intelligence" on December 18, 2023.

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

Paper link: https://www.nature.com/articles/s42256-023-00759-6 needs to be rewritten

Among them, Dr. Liu Shengchao is the first author, and Professor Anima Anandkumar of NVIDIA Research is the corresponding author. Nie Weili, Wang Chengpeng, Lu Jiarui, Qiao Zhuoran, Liu Ling, Tang Jian and Xiao Chaowei are co-authors.

This project was conducted by Dr. Liu Shengchao after joining NVIDIA Research in March 2022, under the guidance of Teachers Nie Weili, Teacher Tang Jian, Teacher Xiao Chaowei and Teacher Anima Anandkumar.

Dr. Liu Shengchao said: "Our motivation was to conduct preliminary exploration of LLM and drug discovery, and finally proposed MoleculeSTM."

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

For docking The text is designed to guide molecule editing

The core idea of ​​MoleculeSTM is very simple and direct, that is, the description of molecules can be divided into two categories: internal chemical structure and external function description. Here we use a contrastive pre-training method to align and connect these two types of information. The specific diagram is shown in the figure below

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

#Illustration: MoleculeSTM flow chart.

This alignment of MoleculeSTM has a very good property: when there are some tasks that are difficult to solve in the chemical space, we can transfer them to the natural language space. And natural language tasks will be relatively easier to solve due to its characteristics. Based on this, we designed a wide variety of downstream tasks to verify its effectiveness. Below we discuss several insights in detail.

Features of Natural Language and Large Language Models

In MoleculeSTM, we pose a problem for the first time. We take advantage of the open vocabulary and combinatorial characteristics of natural language

  • Open vocabulary means that we can express all current human knowledge in natural language, so new knowledge that will appear in the future can also be expressed in modern language. Some language is used to summarize and summarize. For example, if a new protein appears, we hope to describe its function in natural language.
  • Compositionality means that in natural language, a complex concept can be jointly expressed by several simple concepts. This is very helpful for tasks such as multi-attribute editing: it is very difficult to edit molecules to meet multiple properties at the same time in chemical space, but we can express multiple properties very easily in natural language.

In our recent work ChatDrug (https://arxiv.org/abs/2305.18090), we explored the conversational properties between natural language and large language models, which are of interest to Friends can go and take a look

Task design derived from features refers to the design of planning and arranging tasks based on the characteristics of the product or system

For existing language- For image tasks, they can be viewed as art-related tasks, such as generating pictures or text. That is, their results are varied and uncertain. However, scientific discoveries are scientific problems that usually have relatively clear results, such as the generation of small molecules with specific functions. This brings greater challenges in task design

In MoleculeSTM (Appendix B), we proposed two guidelines:

  • The first task we consider is to be able to perform calculations and simulations to obtain results. In the future, wet-lab verification results will be considered, but this is not within the scope of the current work.
  • Secondly, we only consider problems with ambiguous results. Specific examples include making a certain molecule more water-soluble or penetrable. Some problems have clear results, such as adding a certain functional group to a certain position in a molecule. We believe that such tasks are simpler and more straightforward for drug and chemistry experts. So it can be used as a proof-of-concept task in the future, but it will not become the main task target.

From this we designed three broad categories of tasks:

  1. Zero-shot structured text retrieval;
  2. Zero-shot text-based molecule editing ;
  3. Molecular property prediction.

We will focus on the second task in the following sections

The qualitative results of molecular editing are restated as follows:

This The task is to input a molecule and a natural language description (such as additional attributes) at the same time, and then hope to output a new molecule described in a composite language text. This is text-guided lead optimization.

The specific method is to use the already trained molecule generation model and our pre-trained MoleculeSTM to learn the alignment of the two latent spaces (latent space), thereby performing latent space interpolation, and then generate it through decoding target molecule. The process diagram is as follows.

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

The content that needs to be rewritten is: a two-stage process diagram of zero-sample text-guided molecule editing

Here we show several groups of molecule editing The qualitative results are restated as follows: (Result details of the remaining downstream tasks can be found in the original paper). We mainly consider four types of molecular editing tasks:

  • Single attribute editing: Editing a single attribute, such as water solubility, penetrability, and the number of hydrogen bond donors and acceptors.
  • Composite attribute editing: Edit multiple attributes at the same time, such as water solubility and the number of hydrogen bond donors.
  • Drug similarity editor: (Appendix D.5) makes the input molecule and the target molecule drug look closer.
  • Neighbor search for patented drugs: For drugs that have been patented, drugs in the process are often reported together. Here we are combining the intermediate drug with the natural language description to see if it can generate the final target drug.
  • binding affinity editor: We selected several ChEMBL assays as targets, with the goal of having higher binding affinity between the input molecules and the targets.

NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery

#Result display: zero-sample text-guided molecule editing. (Note: This is a direct translation of the original sentence into Chinese.)

What is more interesting is the last type of task. We found that MoleculeSTM can indeed perform matching based on the text description of the target protein. Optimization of lead compounds for ligands. (Note: The protein structure information here will only be known after evaluation.)

The above is the detailed content of NVIDIA, Mila, and Caltech jointly release multi-modal molecular structure-text model of LLM combined with drug discovery. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:机器之心. If there is any infringement, please contact admin@php.cn delete
A Comprehensive Guide to ExtrapolationA Comprehensive Guide to ExtrapolationApr 15, 2025 am 11:38 AM

Introduction Suppose there is a farmer who daily observes the progress of crops in several weeks. He looks at the growth rates and begins to ponder about how much more taller his plants could grow in another few weeks. From th

The Rise Of Soft AI And What It Means For Businesses TodayThe Rise Of Soft AI And What It Means For Businesses TodayApr 15, 2025 am 11:36 AM

Soft AI — defined as AI systems designed to perform specific, narrow tasks using approximate reasoning, pattern recognition, and flexible decision-making — seeks to mimic human-like thinking by embracing ambiguity. But what does this mean for busine

Evolving Security Frameworks For The AI FrontierEvolving Security Frameworks For The AI FrontierApr 15, 2025 am 11:34 AM

The answer is clear—just as cloud computing required a shift toward cloud-native security tools, AI demands a new breed of security solutions designed specifically for AI's unique needs. The Rise of Cloud Computing and Security Lessons Learned In th

3 Ways Generative AI Amplifies Entrepreneurs: Beware Of Averages!3 Ways Generative AI Amplifies Entrepreneurs: Beware Of Averages!Apr 15, 2025 am 11:33 AM

Entrepreneurs and using AI and Generative AI to make their businesses better. At the same time, it is important to remember generative AI, like all technologies, is an amplifier – making the good great and the mediocre, worse. A rigorous 2024 study o

New Short Course on Embedding Models by Andrew NgNew Short Course on Embedding Models by Andrew NgApr 15, 2025 am 11:32 AM

Unlock the Power of Embedding Models: A Deep Dive into Andrew Ng's New Course Imagine a future where machines understand and respond to your questions with perfect accuracy. This isn't science fiction; thanks to advancements in AI, it's becoming a r

Is Hallucination in Large Language Models (LLMs) Inevitable?Is Hallucination in Large Language Models (LLMs) Inevitable?Apr 15, 2025 am 11:31 AM

Large Language Models (LLMs) and the Inevitable Problem of Hallucinations You've likely used AI models like ChatGPT, Claude, and Gemini. These are all examples of Large Language Models (LLMs), powerful AI systems trained on massive text datasets to

The 60% Problem — How AI Search Is Draining Your TrafficThe 60% Problem — How AI Search Is Draining Your TrafficApr 15, 2025 am 11:28 AM

Recent research has shown that AI Overviews can cause a whopping 15-64% decline in organic traffic, based on industry and search type. This radical change is causing marketers to reconsider their whole strategy regarding digital visibility. The New

MIT Media Lab To Put Human Flourishing At The Heart Of AI R&DMIT Media Lab To Put Human Flourishing At The Heart Of AI R&DApr 15, 2025 am 11:26 AM

A recent report from Elon University’s Imagining The Digital Future Center surveyed nearly 300 global technology experts. The resulting report, ‘Being Human in 2035’, concluded that most are concerned that the deepening adoption of AI systems over t

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
1 months agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment