Designing antibodies from scratch, Tencent and Peking University teams pre-trained large language models and published them in Nature sub-journal

编辑 | KX

AI 技术在辅助抗体设计方面取得了巨大进步。然而，抗体设计仍然严重依赖于从血清中分离抗原特异性抗体，这是一个资源密集且耗时的过程。

为了解决这个问题，腾讯 AI Lab、北京大学深圳研究生院和西京消化病医院研究团队提出了一种预训练抗体生成大语言模型(PALM-H3)，用于从头生成具有所需抗原结合特异性的人工抗体CDRH3，减少对天然抗体的依赖。

此外，还设计了一个高精度的抗原-抗体结合预测模型 A2binder，将抗原表位序列与抗体序列配对，从而预测结合特异性和亲和力。

总之，该研究建立了一个用于抗体生成和评估的人工智能框架，这有可能显着加速抗体药物的开发。

相关研究以「De novo generation of SARS-CoV-2 antibody CDRH3 with a pre-trained generative large language model」为题，于 8 月 10 日发布在《Nature Communications》上。

论文链接：https://www.nature.com/articles/s41467-024-50903-y

抗体药物，又称单克隆抗体，在生物治疗中发挥着至关重要的作用。通过模仿免疫系统的作用，这些药物可以选择性地针对病毒和癌细胞等致病因子。与传统治疗方法相比，抗体药物是一种更具体、更有效的方法。抗体药物在治疗多种疾病方面已显示出积极的效果。

开发抗体药物是一个复杂的过程，包括从动物源中分离抗体，使其人性化，并优化其亲和力。但抗体药物的开发仍然严重依赖于天然抗体。

蛋白质的序列数据可以看作是一种语言，因此自然语言处理（NLP）领域的大规模预训练模型已被用来学习蛋白质的表征模式。当前已经开发了多种蛋白质语言模型。然而，由于抗体的多样性高和可用的抗原抗体配对数据稀缺，生成对特定抗原表位具有高亲和力的抗体仍然是一项具有挑战性的任务。

为了应对上述挑战，腾讯AI Lab 团队提出了预训练抗体生成大型语言模型PALM-H3，用于优化和生成重链互补决定区3 (CDRH3)，该区域在抗体的特异性和多样性中起着至关重要的作用。

为了评估 PALM-H3 产生的抗体对抗原的亲和力，研究人员结合使用了抗原抗体对接和基于 AI 的方法。

研究人员还开发了用于评估抗体-抗原亲和力的 A2binder。 A2binder 能够实现准确且可推广的亲和力预测，即使对于未知抗原也是如此。

PALM-H3 和 A2Binder 的框架

PALM-H3 和 A2binder 的工作流程和模型框架如下图所示。

图示：PALM-H3 和 A2binder 工作流程概述。 （来源：论文）

PALM-H3 的目的是生成抗体中的从头 CDRH3 序列。 CDRH3 区域在决定抗体对特定抗原序列的结合特异性方面起着最重要的作用。 PALM-H3 是一个类似 transformer 的模型，它使用基于 ESM2 的抗原模型作为编码器，使用抗体 Roformer 作为解码器。研究还构建了 A2binder 来预测人工生成的抗体的结合亲和力。

PALM-H3 和 A2binder 的构建包括三个步骤：首先，研究人员分别在未配对的抗体重链和轻链序列上预训练两个 Roformer 模型。然后，基于预训练的 ESM2、抗体重链 Roformer 和抗体轻链 Roformer 构建 A2binder，并使用配对亲和力数据对其进行训练。最后，使用预训练的 ESM2 和抗体重链 Roformer 构建 PALM-H3，并在配对抗原-CDRH3 数据上对其进行训练，以从头生成 CDRH3。

A2binder 可以准确预测抗原抗体结合概率、亲和力

通过将 A2binder 预测亲和力的能力与几种基线方法进行比较来评估其性能。

A2binder 在亲和力数据集上表现出色，部分原因在于抗体序列的预训练，这使得 A2binder 能够学习这些序列中存在的独特模式。

Illustration: Comparison of the potential capabilities of pre-trained and untrained models and performance comparison of A2Binder versus baseline methods in predicting antibody-antigen binding specificity. (Source: paper)

The results show that A2binder performs better than the baseline model ESM-F (the latter has the same framework, but the pre-trained model is replaced by ESM2) on all antigen-antibody affinity prediction datasets, which shows Pretraining with antibody sequences may be beneficial for related downstream tasks.

To evaluate the model’s performance in predicting affinity values, the researchers also utilized two datasets, 14H and 14L, that contain affinity value labels.

A2binder outperforms all baseline models on both Pearson correlation and Spearman correlation metrics. A2binder achieves a Pearson correlation of 0.642 on the 14H dataset (an improvement of 3%) and 0.683 on the 14L dataset (an improvement of 1%).

However, the performance of A2binder and other baseline models dropped slightly on the 14H and 14L datasets compared to other datasets. This observation is consistent with previous studies.

PALM-H3 excels in generating high-binding probability antibodies

Researchers explored the differences between the antibodies produced by PALM-H3 and natural antibodies. Their sequences were found to differ significantly, but the binding probabilities of the antibodies produced were not significantly affected by these differences. At the same time, their structural differences do lead to a decrease in binding affinity. These results are consistent with previous studies on network analysis of antibody libraries and generation of functional protein sequences.

Illustration: Performance comparison with baseline methods and similarity analysis of artificial and natural antibodies. (Source: paper)

Overall, the results show that PALM-H3 is capable of generating a diverse range of antibody sequences with high binding affinities, although unlike natural antibodies.

In addition, researchers verified the performance of PALM-H3 through ClusPro and SnugDock. PALM-H3 is capable of generating antibodies against a stabilizing peptide in the HR2 region of SARS-CoV-2, the CDRH3 sequence. It generated a novel CDRH3 sequence and validated that the generated sequence GRREAAWALA has improved targeting of antigen-stabilizing peptides compared to the native CDHR3 sequence GKAAGTFDS.

Illustration: A2binder predicted interface energy comparison between selected high-affinity artificial antibodies and natural antibodies against the SARS-CoV-2 spike protein between different variants and computational structure generation methods. (Source: Paper)

Additionally, PALM-H3 is able to generate antibodies with higher affinity against the emerging SARS-CoV-2 variant XBB CDRH3 sequence. The resulting sequence AKDSRTSPLRLDYS has a stronger affinity for XBB than its source, ASEVLDNLRDGYNF.

In addition, PALM-H3 not only overcomes the local optimal pitfalls faced by traditional sequential mutation strategies, but it also generates antibodies with higher antigen-binding affinity compared to the E-EVO approach. This highlights the advantages of PALM-H3 in antibody design, enabling more efficient exploration of sequence space and generation of high-affinity binders targeting specific epitopes.

In vitro experiments

In addition, the researchers also conducted in vitro experiments, including Western blotting, surface plasmon resonance analysis, and pseudovirus neutralization assays, providing key verification for the effectiveness of the PALM-H3 designed antibodies.

Illustration: In vitro test of binding affinity and neutralization of artificial and natural antibodies. (Source: Paper)

Two antibodies generated by PALM-H3 against SARS-CoV-2 wild-type, Alpha, Delta and XBB variant spike proteins achieved higher binding affinities than native antibodies in these experiments and neutralizing effect. The robust empirical results from these wet lab experiments complement computational predictions and analyses, validating the ability of PALM-H3 and A2binder to generate and select potent antibodies with high specificity and affinity for known and novel antigens.

In summary, the proposed PALM-H3 integrates the ability of large-scale antibody pre-training and the effectiveness of global feature fusion, resulting in excellent affinity prediction performance and the ability to design high-affinity antibodies. Furthermore, direct sequence generation and interpretable weight visualization make it an efficient and interpretable tool for designing high-affinity antibodies.

The above is the detailed content of Designing antibodies from scratch, Tencent and Peking University teams pre-trained large language models and published them in Nature sub-journal. For more information, please follow other related articles on the PHP Chinese website!