


On the first anniversary of the release of the "Tiangong" large model, Kunlun Wanwei announced that the "Tiangong 3.0" base model and the "Tiangong SkyMusic" music model have officially launched public beta .
Since AI has allowed humans to achieve the freedom of music creation, even quarrels have become interesting.
In the past, Aran Komatsuzaki, a well-known AI blogger on the generated. You know, in the past, the war of words between these big guys was mainly to post a post, and then you and I would follow up. This time, Aran Komatsuzaki's approach can be said to have played a new trick. I don't know if it was inspired by "Thank you, thank you, I want to diss you".
Like Aran Komatsuzaki, many people at home and abroad who know music but don’t know music are trying to play AI music creation tools such as Suno, and have generated many very interesting music works. .
However, many netizens have reported that Suno sometimes generates Chinese that is not stable, and there are problems such as Chinese songs having an English feel and uncommon words being sung incorrectly.
# . Video address: https://b23.tv/gVqTUOu
So, is there an AI music generation model specifically optimized for Chinese? Kunlun Wanwei’s “Tiangong SkyMusic”, which is open to the public for public testing today, is such a model. The Chinese vocals it generates are clear, authentic, and have no abnormal sounds, and there are no "Broadway-style Chinese songs" and other acclimatization problems. Moreover, it is not only optimized for Mandarin, but also takes care of dialects such as Cantonese, Chengdu, and Beijing. So, how does SkyMusic perform compared to Suno? Horizontal evaluation data shows that SkyMusic is superior in several indicators such as vocal and BGM sound quality, naturalness of vocals, and intelligibility of pronunciation. Its comprehensive performance surpasses Suno V3, becoming China's first music AIGC SOTA The model also made China's self-developed large model technology lead the world in the AIGC field for the first time.The first domestic music SOTA model with unlimited gameplay
In fact, it is very simple to use SkyMusic to generate songs: you only need to enter the song title, lyrics, and select a reference track, and it will generate songs with similar style and vocals.
#If you don’t want to write lyrics yourself, you can also try the “AI writing” function in the lower right corner of the input box. It can be written from the first sentence, and only one sentence is generated at a time. Unsatisfactory sentences can be deleted in time until the entire song is completed.
Of course, you can also try to use "Tiangong 3.0" to write songs. For example, this song "This Site" was written by us using "Tiangong 3.0":
The next step is to select the reference song. This is also the unique feature of SkyMusic, that is, it can generate music based on the sample sound source.
In this step, SkyMusic provides many reference tracks, you can choose from them, or you can choose to upload song files. Here, we have uploaded a song by Luo Tianyi to see how the generation effect is: This ability to generate music based on sample sound sources has greatly enriched the gameplay of SkyMusic. In the user works display area, we can see that "New Man" (the episode of the movie "Zhou Chu Eliminates Three Evils") has five versions including ancient rock and DJ.
During the trial process, we also discovered that, in fact, the music generated by Tiangong SkyMusic covers rap, folk, funk, ancient style, electronic and other genres. In the next step, the team also plans to allow users to generate songs based on hummed melodies, which will be of great help to professionals.
Currently, Tiangong SkyMusic is fully open. You can experience it by downloading the "Tiangong" APP. This is the only publicly available large-scale AI music generation model in China, and its emergence fills the gap in domestic AIGC tools in this field.
Although this model is still in its infancy, it has already allowed many people to experience the joy of music creation. Everyone uses it to transform divine comedies, create second hits, rewrite ancient poems to support education... and develop various new directions for music creation.
Self-developed Sora architecture, technical roadmap has been made public
Tiangong SkyMusic is an end-to-end music generation model, so it feels very simple for us to use it. However, the development of the entire model is not that simple.
First of all, from a technical perspective, SkyMusic chose the large-model music audio generation route, which means that it directly learns and generates audio waveforms instead of using symbolic music generation routes (such as MIDI) to generate musical scores. . This approach allows for integrated end-to-end generation of elements such as instruments, vocals, melodies, volumes, notes, and more, providing a more direct, higher-quality music creation experience. However, this direction is also more difficult and requires high computing power and funds, so there are very few people doing it.
Moreover, in this direction, there are even fewer people who are determined to conquer the field of "human voice Song". Most research focuses on the field of BGM without voices, because there is almost no open information or open source in the former. Models are available for reference.
Withstanding these pressures, Kunlun Wanwei has conducted countless research and development experiments, invested a lot of computing power, and constructed a data set containing 20 million songs (the largest music data set in human history), and finally explored An effective and reproducible solution was developed. Moreover, they also made the technical schematic diagram of this solution public.
Tiangong SkyMusic technical schematic diagram: Large-scale Transformer is responsible for composing music, learning the contextual dependencies of Music Patches, and completing music controllability sex; the Diffusion Transformer is responsible for singing, and the Music Patches are restored to high-quality audio through LDM. This model architecture works extremely well when processing video, audio, and music.
As can be seen from the picture, the framework of SkyMusic is a DiT technology path similar to Sora, but the development time was before Sora came out, so it is inevitable to step on many pitfalls.
For the industry, this schematic diagram is very valuable, because there is no large-scale AI music model company on the market that has disclosed its own technical path, including SUNO, and Kunlun Wanwei is the only one.
The powerful base behind it - Tiangong 3.0
The success of SkyMusic is inseparable from the base model behind it - Tiangong 3.0. Its core upgrade is reflected in "independent thinking". This is reflected in the model's new multi-round search and comprehensive tool calling, chart drawing, research mode, enhancement mode and other functions.
Given a data statistics task, it can not only help you collect all the data, but also write your own code and call various functions to draw charts. Each intermediate step has been dismantled in a clear manner, and subsequent execution requires little human intervention. Even details such as "avoiding label overlap" and "centering text display" have been taken into consideration. This is the embodiment of the ability of "independent thinking".
This improvement in "independent thinking" ability is inseparable from the optimization of "Tiangong 3.0" in semantic understanding, logical reasoning and other aspects. Compared with the previous generation "Tiangong 2.0" MoE large model, "Tiangong 3.0" has amazing performance improvements in areas such as model semantic understanding, logical reasoning, versatility, generalization, uncertainty knowledge, and learning capabilities. The model technical knowledge ability has increased by more than 20%, and the mathematics/reasoning/coding/cultural and creative abilities have increased by more than 30%.
Take the search task as an example. In the "Search Enhancement" mode, given a simple search request, "Tiangong 3.0" will not only give a summary answer, but also refine some important information into charts.
In the "Research" mode, it will also provide an "In-depth Research" module to discuss extended issues not mentioned in the search query, giving you a The feeling of reading a paper. Finally, it also organizes this information into a mind map for quick reference.
On the basis of greatly improved semantic understanding and logical reasoning capabilities, "Tiangong 3.0" also focuses on the model's ability to independently plan, call, and combine external tools and information. Received special training. By independently planning and calling and combining external tools and information, it can help you accurately and efficiently complete various complex needs such as industrial research, product reviews, and information analysis.
This ability to think independently is crucial for large artificial intelligence models. Firstly, this ability enables AI to perform autonomous reasoning in the absence of direct instructions, improving its ability to handle complex problems; secondly, independently thinking AI models can design innovative solutions to meet personalized and scenario-based needs. demand; finally, this ability enables AI to continuously optimize its performance through self-learning and adaptation when encountering novel or changing environments. The cumulative effect of these three aspects has greatly promoted the breadth and depth of application of AI technology, making it more intelligent and efficient in a variety of practical applications.
"Tiangong 3.0" includes many large model capabilities such as AI music, AI search, AI writing, and AI painting. Talking about the 400 billion parameter MoE large model "Tiangong 3.0" and the technology behind Tiangong SkyMusic At the time of the relationship, Kunlun Wanwei Chairman and CEO Fang Han explained, "Everyone knows that the AI base large model is the solid foundation of AIGC, especially the text large model. Vincent Picture, Vincent Music and Vincent Video (these AIGC models) are the basis of capabilities They are all large text models. If the text model capability is not strong enough, AIGC capabilities will be greatly limited."
This effect is also reflected in the AI painting and other capabilities of "Tiangong 3.0". "Tiangong 3.0" has added new functions such as image size expansion, image orientation adjustment, mat image generation, mat image evolution, and mat image expansion, and the actual test results are excellent.
"Our 400 billion large model is the base model that provides support for all our C-end products. The better my base model is, the better my base model will be. Music, games, videos and animation products will do better, so we have a very strong motivation to make large base models." Fang Han said.
Realize general artificial intelligence,
Let everyone better shape and express themselves
In our vision about AGI, we I often hear some AI company executives mention that they want to use AI tools to improve the productivity and efficiency of human society. Therefore, they mostly focus on the expansion and enhancement of model intelligence. But in Fang Han’s view, an important issue has been overlooked, that is, how to use AI to help people better understand and express emotions.
In the music section of SkyMusic, we have seen many examples of this: the sorrow of students about to graduate, the sorrow of young people unable to love, and the fatigue of middle-aged people supporting their families are all expressed through singing. This is the real "singing to express one's ambition".
The support for dialects is more like a kind of cultural equality, which is Fang Han A very important point. In the future, they hope to add more languages so that every person speaking a small language can easily create their own cultural content.
"In the field of AIGC, our ambitious goal is to hope that everyone in the world can create content equally. We want to lower the creative threshold for everyone, so that everyone can better shape and express Self." Fang Han said.
Recently, these contents have also been written into Kunlun Wanwei’s latest mission.
In fact, this approach also makes sense commercially. "Once everyone can create music, I believe that any public place, such as every restaurant and bar, can create their own background music to meet their own business needs." Fang Han explained.
With continued optimization and improvement in the future, SkyMusic will gradually evolve into a professional and easy-to-use national music creation platform.
Of course, Kunlun Wanwei’s efforts are not limited to music. Based on “Tiangong 3.0”, they have formed six major AI business matrices. In the future, these matrices will form an AI UGC platform.
#This platform can not only help ordinary people express themselves, but also help creators who want to use AI to create content to complete a fully closed loop of IP creation. This closed loop takes "good stories (IP)" as the core and spans various forms such as text, comics, music, and videos. Consumers' content consumption is also completed on this platform. This is the business logic built by Kunlun Wanwei .
"Our essence is to allow more people to join the team of creators. The premise is that you must be able to tell a good story. If you can create a good IP, then you can create content. Yes. It is foreseen that all content industries will be redone. The number of creators will expand a hundred times, and the content that can be consumed will also expand a hundred times. This is why we propose "All in AGI and AIGC". The logic of strategy." Fang Han said.
What will this era be shaped into? We'll see.
The above is the detailed content of The first domestic music SOTA model is here! Optimized specifically for Chinese, free to use, no restrictions on genres. For more information, please follow other related articles on the PHP Chinese website!

你可能听过以下犀利的观点:1.跟着NVIDIA的技术路线,可能永远也追不上NVIDIA的脚步。2.DSA或许有机会追赶上NVIDIA,但目前的状况是DSA濒临消亡,看不到任何希望另一方面,我们都知道现在大模型正处于风口位置,业界很多人想做大模型芯片,也有很多人想投大模型芯片。但是,大模型芯片的设计关键在哪,大带宽大内存的重要性好像大家都知道,但做出来的芯片跟NVIDIA相比,又有何不同?带着问题,本文尝试给大家一点启发。纯粹以观点为主的文章往往显得形式主义,我们可以通过一个架构的例子来说明Sam

2021年9月25日,阿里云发布了开源项目通义千问140亿参数模型Qwen-14B以及其对话模型Qwen-14B-Chat,并且可以免费商用。Qwen-14B在多个权威评测中表现出色,超过了同等规模的模型,甚至有些指标接近Llama2-70B。此前,阿里云还开源了70亿参数模型Qwen-7B,仅一个多月的时间下载量就突破了100万,成为开源社区的热门项目Qwen-14B是一款支持多种语言的高性能开源模型,相比同类模型使用了更多的高质量数据,整体训练数据超过3万亿Token,使得模型具备更强大的推

在法国巴黎举行了国际计算机视觉大会ICCV(InternationalConferenceonComputerVision)本周开幕作为全球计算机视觉领域顶级的学术会议,ICCV每两年召开一次。ICCV的热度一直以来都与CVPR不相上下,屡创新高在今天的开幕式上,ICCV官方公布了今年的论文数据:本届ICCV共有8068篇投稿,其中有2160篇被接收,录用率为26.8%,略高于上一届ICCV2021的录用率25.9%在论文主题方面,官方也公布了相关数据:多视角和传感器的3D技术热度最高在今天的开

8月31日,文心一言首次向全社会全面开放。用户可以在应用商店下载“文心一言APP”或登录“文心一言官网”(https://yiyan.baidu.com)进行体验据报道,百度计划推出一系列经过全新重构的AI原生应用,以便让用户充分体验生成式AI的理解、生成、逻辑和记忆等四大核心能力今年3月16日,文心一言开启邀测。作为全球大厂中首个发布的生成式AI产品,文心一言的基础模型文心大模型早在2019年就在国内率先发布,近期升级的文心大模型3.5也持续在十余个国内外权威测评中位居第一。李彦宏表示,当文心

保险行业对于社会民生和国民经济的重要性不言而喻。作为风险管理工具,保险为人民群众提供保障和福利,推动经济的稳定和可持续发展。在新的时代背景下,保险行业面临着新的机遇和挑战,需要不断创新和转型,以适应社会需求的变化和经济结构的调整近年来,中国的保险科技蓬勃发展。通过创新的商业模式和先进的技术手段,积极推动保险行业实现数字化和智能化转型。保险科技的目标是提升保险服务的便利性、个性化和智能化水平,以前所未有的速度改变传统保险业的面貌。这一发展趋势为保险行业注入了新的活力,使保险产品更贴近人民群众的实际

随着智慧司法的兴起,智能化方法驱动的智能法律系统有望惠及不同群体。例如,为法律专业人员减轻文书工作,为普通民众提供法律咨询服务,为法学学生提供学习和考试辅导。由于法律知识的独特性和司法任务的多样性,此前的智慧司法研究方面主要着眼于为特定任务设计自动化算法,难以满足对司法领域提供支撑性服务的需求,离应用落地有不小的距离。而大型语言模型(LLMs)在不同的传统任务上展示出强大的能力,为智能法律系统的进一步发展带来希望。近日,复旦大学数据智能与社会计算实验室(FudanDISC)发布大语言模型驱动的中

不得不说,Llama2的「二创」项目越来越硬核、有趣了。自Meta发布开源大模型Llama2以来,围绕着该模型的「二创」项目便多了起来。此前7月,特斯拉前AI总监、重回OpenAI的AndrejKarpathy利用周末时间,做了一个关于Llama2的有趣项目llama2.c,让用户在PyTorch中训练一个babyLlama2模型,然后使用近500行纯C、无任何依赖性的文件进行推理。今天,在Karpathyllama2.c项目的基础上,又有开发者创建了一个启动Llama2的演示操作系统,以及一个

杭州第19届亚运会不仅是国际顶级体育盛会,更是一场精彩绝伦的中国科技盛宴。本届亚运会中,快手StreamLake与杭州电信深度合作,联合打造智慧观赛新体验,在击剑赛事的转播中,全面应用了快手StreamLake六自由度技术,其中“子弹时间”也是首次应用于击剑项目国际顶级赛事。中国电信杭州分公司智能亚运专班组长芮杰表示,依托快手StreamLake自研的4K3D虚拟运镜视频技术和中国电信5G/全光网,通过赛场内部署的4K专业摄像机阵列实时采集的高清竞赛视频,


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Dreamweaver Mac version
Visual web development tools

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

Zend Studio 13.0.1
Powerful PHP integrated development environment

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.

SublimeText3 English version
Recommended: Win version, supports code prompts!
