New progress in Alipay AI: Released a large multi-modal medical model and launched an AI medical co-construction plan with 20 institutions

On July 5, at the "Trusted Large Model Forum" of the 2024 World Artificial Intelligence Conference, Alipay announced the latest progress in the layout of its AI technology in the medical field: the release of multi-modal medical large models, and at the same time, new launches including medical reliability A variety of digital intelligent solutions such as trust all-in-one machine and trusted cloud. At the event, Alipay also jointly launched an AI medical co-construction plan with 20 institutions including People's Medical Publishing House, Peking University Health Science Center, and Zhejiang Provincial Health Commission to jointly explore AI digital intelligence technology base and innovative application services. Alipay's multi-modal medical model is unveiled. After more than half a year of testing and polishing, the Alipay medical model is officially unveiled. This is also one of the first multi-modal medical models in China. Gu Jinjie, general manager of the Large Model Application Department of Ant Group, introduced the technology research and development behind it. The Ant Bailing large model already has native multi-modal capabilities that can "see", "listen", "speak" and "draw". It can directly understand and train multi-modal data such as audio, video, pictures, text and so on. Alipay's large medical model uses this as a base model and adds tens of billions of Chinese and English pictures and texts including reports, images, medicines, etc., hundreds of billions of medical text corpus, tens of millions of high-quality medical knowledge maps, Chinese and English medical examinations, and benchmarks. The test reaches or exceeds the GPT4 level, and the medical professional ability ranks first in the A list and second in the B list in promptCBLUE, the Chinese medical LLM evaluation list. At the same time, the full-stack self-developed medical multi-modal structure enables the accuracy of model identification reports, drugs, hair and other images to reach more than 90%. The related technology has published many results in the industry's top technical journals and conferences such as CVPR, KDD, ECCV, etc. In order to make answering conversations more reliable, during the research and development stage, Alipay also cooperated with hundreds of professional medical teams and experts to label data to ensure the authority and credibility of the information. At the same time, it launched the first Chinese medical specialty question and answer reasoning data set RJUA in conjunction with Shanghai Renji Hospital -QA. During the model testing phase, users, patients, and doctors participated in question and answer feedback, while maintaining professionalism and caring for patients.

Alipay medical model release site
Currently, this multi-modal medical model not only provides intelligent question and answer, medical record structuring and retrieval, and auxiliary diagnosis, but can also identify and interpret drugs and hundreds of complex medical reports to conduct hair health Testing, etc., these can be embedded in the entire medical process of hospitals and other institutions. While innovating medical services, they can also help improve the operational efficiency of the institution.
Gu Jinjie said that the in-depth application of AI not only puts forward higher requirements for data, algorithms, and computing power, but also poses more challenges to reliability, security, and privacy. In medical scenarios, it is crucial to ensure technical reliability and data privacy security. . To this end, Alipay has launched a new "medical trusted all-in-one machine + trusted cloud" solution for the industry. Through dense computing and storage, medical data is made available and invisible, and institutional storage is safer.
According to reports, Bailing Medical Trusted All-in-one Machine core solves the pain point of insufficient computing power faced by hospitals. It can be used out of the box, integrates training and promotion, supports domestic computing power, and software and hardware collaborative optimization and acceleration. The trusted cloud solution for medical large models deploys dense state reasoning through public clouds and private clouds, focusing on solving data privacy security and business secret protection issues.
The AI ​​Medical Innovation Application “Alliance” is here
Since last year, Alipay has accelerated the opening and deployment of AI technology in the medical field. In November last year, the Zhejiang Health Commission applied the "AI Medical Assistant" solution opened by Alipay and relied on large models and digital human technology to create the country's first digital health person "Anclin" to provide Zhejiang residents with cloud-based medical consultation, Health consultation and other services have served more than 1,000 medical institutions since their launch. Shanghai First People's Hospital, using large AI models, not only launched Shanghai's first voice-interactive "digital attendant", but also pioneered a generative electronic medical record in the industry, reducing the complicated paperwork from the original 10 minutes. It's 15 seconds.
Innovation in medical technology requires the joint efforts of platforms, industries, and academia. At the conference site, Alipay, People's Medical Publishing House, Peking University Health Science Center, Zhejiang Provincial Health Commission, Xiamen Municipal Health Commission, Zhejiang Provincial People's Hospital, Renji Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Peking University People's Hospital, Fudan University Oncology More than 20 institutions and enterprises, including the hospital, Shanghai First People's Hospital, and the Affiliated Hospital of Capital Medical University, jointly launched an AI medical co-construction plan to jointly explore the application of large models and innovative research and development of various specialist models.

The launch site of Alipay’s AI medical co-construction plan

Zhang Junjie, Alipay’s vice president and general manager of the digital health business unit, said:

“Alipay hopes to use the most cutting-edge technology to provide the most inclusive services, so that AI can be like Scanning the QR code to pay will make everyone’s life more convenient. Alipay will continue to open up AI technology and industry solutions, and work with partners to promote the inclusiveness and popularization of smart medical care.”

It is understood that

• In 2014, Alipay supported the first remote registration payment in the country.

• After ten years of development, the platform has served more than 600 million people seeking medical treatment, making it the largest in China:

  • Medical insurance third-party online service platform
  • One-stop medical and health service platform

Currently

• Over 3,600 hospitals in more than 300 cities across the country provide one-stop digital medical services on Alipay.

The above is the detailed content of New progress in Alipay AI: Released a large multi-modal medical model and launched an AI medical co-construction plan with 20 institutions. For more information, please follow other related articles on the PHP Chinese website!