search
HomeTechnology peripheralsAIClone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Previously, the incident of OpenAI not being opened has caused a lot of controversy in the public.

Only releasing benchmarks and test results without providing training data, costs, and methods is really a "winner takes all" situation.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Seeing that large language models seem to be monopolized by giant companies, now a start-up suddenly emerges and gives OpenAI a shot - with 6 billion The parameter "Dolly" implements similar capabilities to ChatGPT.

Yes, we only need to prepare some high-quality training data now, and then randomly pick up a large open source language model. After training for 30 minutes, we can get a ChatGPT "replacement" ”!

In this regard, Databricks proudly stated that the release of Dolly is its first step on the road to democratizing artificial intelligence technology.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

6 billion parameters are comparable to ChatGPT, and can be trained in 30 minutes

Because ChatGPT consumes a lot of data and computing resources (Training using trillions of words consumes a lot of GPU), so this type of large language model is destined to be mastered only by a few giants.

Contrary to "CloseAI", Meta released a set of high-quality (but not instruction-following) language models LLaMA to the academic community in March this year. The training time of each model exceeds 80,000 GPU hours.

Stanford then built Alpaca based on LLaMA, but the difference was that it was fine-tuned using a small data set of 50,000 questions and answers. Surprisingly, this gives Alpaca interactivity similar to ChatGPT.

And Dolly was inspired by Alpaca.

What’s more interesting is that Dolly, which has 6 billion parameters, did not use the latest model, but chose an open source model released in 2021-GPT-J.

Since Dolly itself is a "clone" of a model, the team finally decided to name it "Dolly" - the first cloned animal ever.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Compared with current large language models (such as GPT-3), Dolly allows users to use smaller and more professional models, "Complex The ability to "engrave" ChatGPT.

After all, for those niche users, being able to take advantage of models that have been fine-tuned for their industry can greatly increase performance and accuracy.

Although Databricks does not directly compete with OpenAI, it seems to be trying to steal OpenAI's limelight by proving that building a service like ChatGPT is not as difficult as it seems.

In particular, OpenAI has taken a "bigger is better" approach to developing language models and has become increasingly secretive about its work.

In addition to releasing Dolly as open source software, Databricks also emphasized that Dolly only has 6 billion parameters (the part of the language model that is fine-tuned during training), while OpenAI’s GPT-3 model has 175 billion parameters. (OpenAI did not disclose the number of parameters for GPT-4).

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Let the old model be reborn

After evaluating Dolly based on the instruction following ability described in the InstructGPT paper, it was found that Its performance is very similar to ChatGPT in many capabilities, including text generation, brainstorming, and open question and answer.

What is noteworthy in these examples is not the quality of the generated text, but the huge improvement in the ability to follow instructions brought by fine-tuning an old open source model on a small high-quality dataset.

Content generation

#For example, write a tweet about the official announcement of Databricks’ large-scale language model Dolly.

It can be seen that the content generated by the original 6 billion parameter model (GPT-J) is completely incorrect, while Dolly gave a completely usable tweet— —

Not only does the content meet the requirements, but it also thoughtfully adds tags and links to remind you to join the publication.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

For this question, the answer given by ChatGPT is also in line with expectations. Compared with Dolly, the tweet given by ChatGPT contains more There are more descriptive words and sentences, and the labels are more precise and specific, but the overall difference is not big.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

When you want to write an advertisement for selling a Nikon D-750 camera, you can see that the content generated by GPT-J is basically Making up random stories about buying and selling cameras like writing a novel...

And Dolly gave an attractive story based on the characteristics and advantages of the Nikon D-750 camera. Human camera resale slogan, but unfortunately the pixel parameters are wrong.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

ChatGPT has also successfully completed the task on this issue. The advertising slogan highlights the advantages of this camera, and the tag is still thoughtfully added at the end of the article. . ​

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

The last question: Write a book to Edgar Allan Poe.

In this regard, the ancient GPT-J directly refused to answer. The reason turned out to be - Edgar Allan Poe has passed away, and you cannot write love letters to the dead.

And Dolly successfully completed the task, and the effect can be called "Nirvana" in comparison.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

And this kind of "creative" problem is obviously ChatGPT's strength. It wrote more than 300 words eloquently.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Open Q&A

In the question and answer test of factual questions, the team chose the following: "Explain to me the difference between nuclear fission and nuclear fusion."

Regardless of whether it is right or wrong, the entire article of GPT-J is about the sun, although it mentions " The word "fusion" is used, but "fission" is completely ignored.

And Dolly directly pointed out the topic in the first sentence - the difference between nuclear fission and nuclear fusion lies in the way of releasing energy, and then briefly explained their differences.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

In contrast, the answers given by ChatGPT are obviously more informative.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

Brainstorming

When asked to brainstorm a list of five science fiction novels they should read, GPT-J just muttered to themselves, like I was immersed in the guilt caused by procrastinating reading, and completely avoided this question.

Dolly performed as steadily as ever and followed the instructions to give the titles of five science fiction novels and their authors.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

ChatGPT gave a richer answer to this question, including not only the title and author of the book, but also the content and type of each book A brief review and introduction were given.

Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5

If you want to Close, I will Open

For many companies, they would rather build a less strong model themselves. You also don't want to send data to big language model vendors who only provide APIs.

One of the important reasons is that these questions and data sets are the company's most sensitive and proprietary intellectual property, and it is obviously unreliable to hand them over directly to a third party.

In addition, companies themselves may have different trade-offs in terms of model quality, cost, and desired behavior, and a customizable language model is more in line with their needs.

Now, the release of Dolly gives them hope that even an "outdated" open source large language model (LLM) can be trained for 30 minutes to give it magical Similar to ChatGPT's command following ability.

It is not difficult to imagine that large language models may soon no longer be exclusive to AI giants!

As the company’s CEO Ali Ghodsi said, “Our belief is that every organization in the world can take advantage of these technologies.”

The above is the detailed content of Clone ChatGPT with zero threshold! After 30 minutes of training, the performance of 6 billion parameters is comparable to GPT-3.5. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
You Must Build Workplace AI Behind A Veil Of IgnoranceYou Must Build Workplace AI Behind A Veil Of IgnoranceApr 29, 2025 am 11:15 AM

In John Rawls' seminal 1971 book The Theory of Justice, he proposed a thought experiment that we should take as the core of today's AI design and use decision-making: the veil of ignorance. This philosophy provides a simple tool for understanding equity and also provides a blueprint for leaders to use this understanding to design and implement AI equitably. Imagine that you are making rules for a new society. But there is a premise: you don’t know in advance what role you will play in this society. You may end up being rich or poor, healthy or disabled, belonging to a majority or marginal minority. Operating under this "veil of ignorance" prevents rule makers from making decisions that benefit themselves. On the contrary, people will be more motivated to formulate public

Decisions, Decisions… Next Steps For Practical Applied AIDecisions, Decisions… Next Steps For Practical Applied AIApr 29, 2025 am 11:14 AM

Numerous companies specialize in robotic process automation (RPA), offering bots to automate repetitive tasks—UiPath, Automation Anywhere, Blue Prism, and others. Meanwhile, process mining, orchestration, and intelligent document processing speciali

The Agents Are Coming – More On What We Will Do Next To AI PartnersThe Agents Are Coming – More On What We Will Do Next To AI PartnersApr 29, 2025 am 11:13 AM

The future of AI is moving beyond simple word prediction and conversational simulation; AI agents are emerging, capable of independent action and task completion. This shift is already evident in tools like Anthropic's Claude. AI Agents: Research a

Why Empathy Is More Important Than Control For Leaders In An AI-Driven FutureWhy Empathy Is More Important Than Control For Leaders In An AI-Driven FutureApr 29, 2025 am 11:12 AM

Rapid technological advancements necessitate a forward-looking perspective on the future of work. What happens when AI transcends mere productivity enhancement and begins shaping our societal structures? Topher McDougal's upcoming book, Gaia Wakes:

AI For Product Classification: Can Machines Master Tax Law?AI For Product Classification: Can Machines Master Tax Law?Apr 29, 2025 am 11:11 AM

Product classification, often involving complex codes like "HS 8471.30" from systems such as the Harmonized System (HS), is crucial for international trade and domestic sales. These codes ensure correct tax application, impacting every inv

Could Data Center Demand Spark A Climate Tech Rebound?Could Data Center Demand Spark A Climate Tech Rebound?Apr 29, 2025 am 11:10 AM

The future of energy consumption in data centers and climate technology investment This article explores the surge in energy consumption in AI-driven data centers and its impact on climate change, and analyzes innovative solutions and policy recommendations to address this challenge. Challenges of energy demand: Large and ultra-large-scale data centers consume huge power, comparable to the sum of hundreds of thousands of ordinary North American families, and emerging AI ultra-large-scale centers consume dozens of times more power than this. In the first eight months of 2024, Microsoft, Meta, Google and Amazon have invested approximately US$125 billion in the construction and operation of AI data centers (JP Morgan, 2024) (Table 1). Growing energy demand is both a challenge and an opportunity. According to Canary Media, the looming electricity

AI And Hollywood's Next Golden AgeAI And Hollywood's Next Golden AgeApr 29, 2025 am 11:09 AM

Generative AI is revolutionizing film and television production. Luma's Ray 2 model, as well as Runway's Gen-4, OpenAI's Sora, Google's Veo and other new models, are improving the quality of generated videos at an unprecedented speed. These models can easily create complex special effects and realistic scenes, even short video clips and camera-perceived motion effects have been achieved. While the manipulation and consistency of these tools still need to be improved, the speed of progress is amazing. Generative video is becoming an independent medium. Some models are good at animation production, while others are good at live-action images. It is worth noting that Adobe's Firefly and Moonvalley's Ma

Is ChatGPT Slowly Becoming AI's Biggest Yes-Man?Is ChatGPT Slowly Becoming AI's Biggest Yes-Man?Apr 29, 2025 am 11:08 AM

ChatGPT user experience declines: is it a model degradation or user expectations? Recently, a large number of ChatGPT paid users have complained about their performance degradation, which has attracted widespread attention. Users reported slower responses to models, shorter answers, lack of help, and even more hallucinations. Some users expressed dissatisfaction on social media, pointing out that ChatGPT has become “too flattering” and tends to verify user views rather than provide critical feedback. This not only affects the user experience, but also brings actual losses to corporate customers, such as reduced productivity and waste of computing resources. Evidence of performance degradation Many users have reported significant degradation in ChatGPT performance, especially in older models such as GPT-4 (which will soon be discontinued from service at the end of this month). this

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),