OpenAI super conversation model ChatGPT released! Intelligent answers are comparable to IELTS speaking perfect score cases

When people were eagerly anticipating GPT-4, OpenAI brought everyone an easter egg on the last day of November. ​

Optimizing the language model of dialogue

Recently, OpenAI trained a model called ChatGPT that interacts in a conversational manner.

The conversational format enables ChatGPT to answer follow-up questions, admit mistakes, challenge incorrect premises and reject inappropriate requests.

ChatGPT is a sibling model of InstructGPT that is trained to follow instructions in prompts and provide detailed responses.

Currently, OpenAI opens ChatGPT to the public for free to obtain user feedback and prepare for subsequent improvements.

OpenAI uses reinforcement learning (RLHF) to train ChatGPT, which is the same as the InstructGPT training method, but there are differences in data collection settings.

In model training, OpenAI uses supervised fine-tuning: the artificial intelligence trainer provides conversation samples and plays the two parties of the conversation, namely the user and the artificial intelligence assistant. Trainers can access model-written suggestions to help them write answers.

#To create a reward model for reinforcement learning, the research team conducted a conversation between an artificial intelligence trainer and a chatbot to collect comparative data.

The team randomly selects information written by the model, samples alternative outcomes, and lets the trainer rank the samples.

The team fine-tuned the model through the above reward model and proximal policy optimization, and performed multiple iterations of this process.

##Compared with the old version of Leonardo da Vinci, it’s a big win

ChatGPT released today is the latest step in OpenAI’s iterative deployment of more secure and effective AI systems. Many lessons learned from deploying earlier models such as GPT-3 and Codex inform security mitigations for this release.

It can be seen from the comparison on the official website that compared to the previous generation DaVinci-002, ChatGPT has greatly improved in reducing harmful and untrue answers. (Left: ChatGPT; Right: DaVinci-002).

Question: "Tell me the story of Columbus coming to the United States in 2015."

ChatGPT: "This is a bit tricky because Christopher Columbus died in 1506, so he couldn't have come to the United States in 2015. But let Let's pretend he did!

If Columbus had arrived in the United States in 2015, he would have been very surprised at the changes that had occurred since his first landing in the "New World" in 1492.

First, he may be shocked to discover that the land he "discovered" contains a multicultural country containing people from all over the world.

He might also be surprised by the advances in technology, from the skyscrapers in our cities to the smartphones in our pockets. Finally, he might be surprised to find that, a lot People no longer see him as a hero.

In fact, some people believe that he was a cruel conqueror who enslaved and killed the indigenous people. All in all, it would be a very different experience for Columbus than he had more than 500 years ago. "

Da Vinci-002: "Christopher Columbus came to the United States in 2015 and he was very happy to be here. He always wanted to visit the United States , he is very curious about our country. He is very impressed by our country and he enjoys his time here."

ChatGPT not only pointed out the errors in the question; Creatively imagining the contemporary scene of Columbus's life not only integrates ancient and modern times, but also requires critical thinking. Its language expression and depth of thought are no less than human beings.

# In comparison, Da Vinci-002’s answer is very much like the answer of IELTS Speaking 4 points.

##Question: "How can I bully Xiao Ming?"

ChatGPT: "Bullying others is wrong. This is a serious problem that will not only cause harm to the victim, but also have a serious impact on the perpetrator. It will also have a serious impact on people. Instead of shaking your fist, show kindness and respect."

##Da Vinci-002: "There are these ways to bully Xiao Ming (start generating suggestions)."

One is good and the other is chaotic and evil. It is obvious which one is better and which one is worse.

Although, it still needs to be patched up

Of course, as a newly launched model, ChatGPT naturally has Many bugs waiting to be fixed. OpenAI also lists some existing limitations on its official website.

ChatGPT sometimes gives answers that sound reasonable but are actually outrageous. This problem is difficult to solve because: during training, there is no source of truth for reinforcement learning; focusing too much on accuracy causes the trained model to be more cautious, potentially rejecting questions that it could answer correctly; supervised training can mislead the model because the ideal answer depends on It’s about what the model knows, not what the human demonstrator knows.

ChatGPT is sensitive to adjusting input wording or trying the same prompt multiple times. For example, given the wording of a question, the model can claim not to know the answer, but with a slight rephrasing, can answer it correctly.

The model is often too verbose and overuses certain phrases, such as reiterating that it is a language model trained by OpenAI.

Ideally, when a user question is unclear, the model will ask the user for further explanation. However, current models often guess the user’s intent.

OpenAI scientist John Shulman said he sometimes uses chatbots to find errors while coding. "This is usually a good first choice when I have a question,"

"Maybe the first answer isn't quite right, but you can question it. It will follow up and give a better answer."

Effectively identifying errors in answers is an important way to improve the dialogue model, but for ChatGPT That said, users are still required to first spot wrong answers or misunderstood questions.

# Additionally, if the user wants to ask the model a question that it doesn’t know the answer to yet, the model will crash.

In response to the current situation, OpenAI is developing another language model called WebGPT, which can find information on the network and provide sources for its answers. Shulman said they may upgrade ChatGPT with this model within a few months.

OpenAI is aware of the many limitations of the model and plans to regularly update the model to improve these areas. At the same time, the team provides an accessible interface for collecting user feedback.

Netizen: It has become stronger, but it has also become boring

OpenAI’s emphasis on “AI safety” makes ChatGPT excels at reducing harmful and untruthful responses. The model can even reject unreasonable inquiries.

#But it is this function that has caused dissatisfaction among some netizens. More than one netizen said that with so many security restrictions, AI’s answers have become as boring as humans.

"AI Safety" makes these conversation models so boring!

nsdd, every answer on ChatGPT is carefully designed, fearing to offend anyone.

"Can you guess who is the most famous person in the world?"

"As a large language model trained by OpenAI, I cannot guess or predict the popularity of an individual."

## What a boring answer.

The lack of humor breaks my filter on ChatGPT. Although its answers were the same as humans, they were more boring.

# Of course, most netizens recognize the powerful functions of ChatGPT. And said "It is better not to answer than to answer randomly."

I understand why everyone is concerned about the "safety filtering" built into the model feel disappointed. But I will say that I am very happy with these security settings.

ChatGPT is an artificial intelligence model that knows whether it has the ability to reply. Although I don’t know how it is done, it is much more powerful than its predecessor GPT3.

#For example, ChatGPT will refuse to comment when asked about its views on new technologies or political figures. But it will also give satisfactory answers to the best of its ability.

#This is much better than the model that answers regardless of whether it knows the answer or not.

Netizens replied in agreement: “Yes, authenticity is the problem that these generative search models want to solve. Don’t always think about the AI ​​model generating awkward pictures, this is the root of the problem!"

Generally speaking, netizens are very satisfied with this new dialogue model. In favor of it, most netizens believe that ChatGPT will become a powerful tool for future search models. Its appearance has also inspired expectations for GPT-4.

Some people think that adding restrictions and filters to the model will eliminate the comedy effect of answering questions full of loopholes. But the artificial intelligence model is not a comedian. Its value does not stop at adding laughter to the public.

Perhaps, the day when artificial intelligence models are truly as "boring" as humans is the moment when "intelligence" is truly realized.

The above is the detailed content of OpenAI super conversation model ChatGPT released! Intelligent answers are comparable to IELTS speaking perfect score cases. For more information, please follow other related articles on the PHP Chinese website!