


Previously, the incident of OpenAI not being opened has caused a lot of controversy in the public.
Only releasing benchmarks and test results without providing training data, costs, and methods is really a "winner takes all" situation.
Seeing that large language models seem to be monopolized by giant companies, now a start-up suddenly emerges and gives OpenAI a shot - with 6 billion The parameter "Dolly" implements similar capabilities to ChatGPT.
Yes, we only need to prepare some high-quality training data now, and then randomly pick up a large open source language model. After training for 30 minutes, we can get a ChatGPT "replacement" ”!
In this regard, Databricks proudly stated that the release of Dolly is its first step on the road to democratizing artificial intelligence technology.
6 billion parameters are comparable to ChatGPT, and can be trained in 30 minutes
Because ChatGPT consumes a lot of data and computing resources (Training using trillions of words consumes a lot of GPU), so this type of large language model is destined to be mastered only by a few giants.
Contrary to "CloseAI", Meta released a set of high-quality (but not instruction-following) language models LLaMA to the academic community in March this year. The training time of each model exceeds 80,000 GPU hours.
Stanford then built Alpaca based on LLaMA, but the difference was that it was fine-tuned using a small data set of 50,000 questions and answers. Surprisingly, this gives Alpaca interactivity similar to ChatGPT.
And Dolly was inspired by Alpaca.
What’s more interesting is that Dolly, which has 6 billion parameters, did not use the latest model, but chose an open source model released in 2021-GPT-J.
Since Dolly itself is a "clone" of a model, the team finally decided to name it "Dolly" - the first cloned animal ever.
Compared with current large language models (such as GPT-3), Dolly allows users to use smaller and more professional models, "Complex The ability to "engrave" ChatGPT.
After all, for those niche users, being able to take advantage of models that have been fine-tuned for their industry can greatly increase performance and accuracy.
Although Databricks does not directly compete with OpenAI, it seems to be trying to steal OpenAI's limelight by proving that building a service like ChatGPT is not as difficult as it seems.
In particular, OpenAI has taken a "bigger is better" approach to developing language models and has become increasingly secretive about its work.
In addition to releasing Dolly as open source software, Databricks also emphasized that Dolly only has 6 billion parameters (the part of the language model that is fine-tuned during training), while OpenAI’s GPT-3 model has 175 billion parameters. (OpenAI did not disclose the number of parameters for GPT-4).
Let the old model be reborn
After evaluating Dolly based on the instruction following ability described in the InstructGPT paper, it was found that Its performance is very similar to ChatGPT in many capabilities, including text generation, brainstorming, and open question and answer.
What is noteworthy in these examples is not the quality of the generated text, but the huge improvement in the ability to follow instructions brought by fine-tuning an old open source model on a small high-quality dataset.
Content generation
#For example, write a tweet about the official announcement of Databricks’ large-scale language model Dolly.
It can be seen that the content generated by the original 6 billion parameter model (GPT-J) is completely incorrect, while Dolly gave a completely usable tweet— —
Not only does the content meet the requirements, but it also thoughtfully adds tags and links to remind you to join the publication.
For this question, the answer given by ChatGPT is also in line with expectations. Compared with Dolly, the tweet given by ChatGPT contains more There are more descriptive words and sentences, and the labels are more precise and specific, but the overall difference is not big.
When you want to write an advertisement for selling a Nikon D-750 camera, you can see that the content generated by GPT-J is basically Making up random stories about buying and selling cameras like writing a novel...
And Dolly gave an attractive story based on the characteristics and advantages of the Nikon D-750 camera. Human camera resale slogan, but unfortunately the pixel parameters are wrong.
ChatGPT has also successfully completed the task on this issue. The advertising slogan highlights the advantages of this camera, and the tag is still thoughtfully added at the end of the article. .
The last question: Write a book to Edgar Allan Poe.
In this regard, the ancient GPT-J directly refused to answer. The reason turned out to be - Edgar Allan Poe has passed away, and you cannot write love letters to the dead.
And Dolly successfully completed the task, and the effect can be called "Nirvana" in comparison.
And this kind of "creative" problem is obviously ChatGPT's strength. It wrote more than 300 words eloquently.
Open Q&A
In the question and answer test of factual questions, the team chose the following: "Explain to me the difference between nuclear fission and nuclear fusion."
Regardless of whether it is right or wrong, the entire article of GPT-J is about the sun, although it mentions " The word "fusion" is used, but "fission" is completely ignored.
And Dolly directly pointed out the topic in the first sentence - the difference between nuclear fission and nuclear fusion lies in the way of releasing energy, and then briefly explained their differences.
In contrast, the answers given by ChatGPT are obviously more informative.
Brainstorming
When asked to brainstorm a list of five science fiction novels they should read, GPT-J just muttered to themselves, like I was immersed in the guilt caused by procrastinating reading, and completely avoided this question.
Dolly performed as steadily as ever and followed the instructions to give the titles of five science fiction novels and their authors.
ChatGPT gave a richer answer to this question, including not only the title and author of the book, but also the content and type of each book A brief review and introduction were given.
If you want to Close, I will Open
For many companies, they would rather build a less strong model themselves. You also don't want to send data to big language model vendors who only provide APIs.
One of the important reasons is that these questions and data sets are the company's most sensitive and proprietary intellectual property, and it is obviously unreliable to hand them over directly to a third party.
In addition, companies themselves may have different trade-offs in terms of model quality, cost, and desired behavior, and a customizable language model is more in line with their needs.
Now, the release of Dolly gives them hope that even an "outdated" open source large language model (LLM) can be trained for 30 minutes to give it magical Similar to ChatGPT's command following ability.
It is not difficult to imagine that large language models may soon no longer be exclusive to AI giants!
As the company’s CEO Ali Ghodsi said, “Our belief is that every organization in the world can take advantage of these technologies.”
The above is the detailed content of Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5. For more information, please follow other related articles on the PHP Chinese website!

自从 ChatGPT、Stable Diffusion 发布以来,各种相关开源项目百花齐放,着实让人应接不暇。今天,着重挑选几个优质的开源项目分享给大家,对我们的日常工作、学习生活,都会有很大的帮助。

Word文档拆分后的子文档字体格式变了的解决办法:1、在大纲模式拆分文档前,先选中正文内容创建一个新的样式,给样式取一个与众不同的名字;2、选中第二段正文内容,通过选择相似文本的功能将剩余正文内容全部设置为新建样式格式;3、进入大纲模式进行文档拆分,操作完成后打开子文档,正文字体格式就是拆分前新建的样式内容。

用 ChatGPT 辅助写论文这件事,越来越靠谱了。 ChatGPT 发布以来,各个领域的从业者都在探索 ChatGPT 的应用前景,挖掘它的潜力。其中,学术文本的理解与编辑是一种极具挑战性的应用场景,因为学术文本需要较高的专业性、严谨性等,有时还需要处理公式、代码、图谱等特殊的内容格式。现在,一个名为「ChatGPT 学术优化(chatgpt_academic)」的新项目在 GitHub 上爆火,上线几天就在 GitHub 上狂揽上万 Star。项目地址:https://github.com/

阅读论文可以说是我们的日常工作之一,论文的数量太多,我们如何快速阅读归纳呢?自从ChatGPT出现以后,有很多阅读论文的服务可以使用。其实使用ChatGPT API非常简单,我们只用30行python代码就可以在本地搭建一个自己的应用。 阅读论文可以说是我们的日常工作之一,论文的数量太多,我们如何快速阅读归纳呢?自从ChatGPT出现以后,有很多阅读论文的服务可以使用。其实使用ChatGPT API非常简单,我们只用30行python代码就可以在本地搭建一个自己的应用。使用 Python 和 C

面对一夜爆火的 ChatGPT ,我最终也没抵得住诱惑,决定体验一下,不过这玩意要注册需要外国手机号以及科学上网,将许多人拦在门外,本篇博客将体验当下爆火的 ChatGPT 以及无需注册和科学上网,拿来即用的 ChatGPT 使用攻略,快来试试吧!

ChatGPT可以联网后,OpenAI还火速介绍了一款代码生成器,在这个插件的加持下,ChatGPT甚至可以自己生成机器学习模型了。 上周五,OpenAI刚刚宣布了惊爆的消息,ChatGPT可以联网,接入第三方插件了!而除了第三方插件,OpenAI也介绍了一款自家的插件「代码解释器」,并给出了几个特别的用例:解决定量和定性的数学问题;进行数据分析和可视化;快速转换文件格式。此外,Greg Brockman演示了ChatGPT还可以对上传视频文件进行处理。而一位叫Andrew Mayne的畅销作

本篇文章给大家带来了关于php的相关知识,其中主要介绍了我是怎么用ChatGPT学习PHP中AOP的实现,感兴趣的朋友下面一起来看一下吧,希望对大家有帮助。


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft
