What is ChatGPT? What do G, P, and T stand for?

Bill Gates: ChatGPT is the most revolutionary technological advancement since 1980.

In this era of AI transformation, we can only join in and keep up.

This is my study note, I hope it will be helpful for you to understand ChatGPT.

1. What do the GPTs in ChatGPT stand for?

GPT, Generative Pre-trained Transformer, generative pre-trained transformation model.

What's the meaning?

Generative means that it can generate content spontaneously.

Pre-trained, pre-training, does not require you to get it and train again. It directly prepares a general language model for you.

Transformer, transformation model, is a very powerful model proposed by Google. It can help better deal with NLP related problems. It is a very good neural network structure.

2. Although Transformer was proposed by Google. But the most successful application is OpenAI’s ChatGPT.

Because ChatGPT stands on the shoulders of giants.

ChatGPT is the crystallization of all human society. Without the paving the way, ChatGPT would not have come out so smoothly.

There will be more very powerful applications in the future, which will be based on ChatGPT.

3. There is a very important concept in Transformer, the attention mechanism.

What is the attention mechanism?

It is to pick out the important information from the information you input, focus on this important information, and ignore the unimportant information. This will help you better understand what you are saying.

The attention mechanism can help the Transformer model focus on the most important part of the input information.

4. Machine learning methods are divided into supervised learning, unsupervised learning, and reinforcement learning.

Supervised learning: There is labeled data, direct feedback can be given, and results and the future can be predicted.

Unsupervised learning: There are no labels, goals, and no feedback, but it searches for the hidden meanings in the data by itself. Results

Reinforcement learning: It is a decision-making process with a series of reward mechanisms and punishment mechanisms to help machine learning do better

ChatGPT uses unsupervised learning and reinforcement learning.

5. ChatGPT can generate and create a large amount of content, but it actually relies on guessing probability.

For example, the weather is gray and my mood is very___

The AI ​​trained with a large amount of data will predict that the word with the highest probability of appearing in this space is "depressed".

Then "depressed" will be filled in this blank, so the answer is:

The weather is gray and gray, and I feel very depressed

This feels incredible , but that’s the fact.

All NLP (natural language processing) tasks at this stage are not machines that can truly understand the human world.

He was just playing word games and solving probability puzzles again and again.

#6. In this word game of "guessing the probability", the large prediction model (LLM, Large Language Model) has evolved into the two most mainstream directions: BERT and GPT.

BERT was the most popular direction before, dominating almost all NLP fields.

And perform well in natural language understanding tasks (such as text classification, emotional tendency judgment, etc.).

The GPT direction is relatively weak, and the most well-known player is OpenAl.

In fact, before the release of GPT3.0, the direction of GPT has always been weaker than BERT (GPT3.0 is the predecessor of GPT3.5, the model behind ChatGPT).

7. What is the difference between BERT and GPT?

BERT is a two-way language model. It guesses the word in the middle before and after the connection, so it is two-way, just like cloze.

For example: I ___go home on the 20th

BERT guessed "I plan to go home on the 20th" and guessed the "plan" in the middle.

GPT is a one-way language model, guessing the next word, so it is one-way, like writing a composition.

For example: I plan to go home on the 20th___

GPT guessed "I plan to go home on the 20th", and guessed the word "go home" after it.

8. How to ask questions to GPT?

There are two ways: fine-tune and prompt.

fine-tune, parameter adjustment: The model parameters need to be updated to complete the generated content.

fine-tune is professional, with high threshold and small audience. However, it has high diversity and accuracy and is suitable for complex tasks. A game for a few players.

prompt, prompt word: No need to modify the model and parameters, just give some tips and examples to complete the task.

prompt is simpler, the threshold is low, and the audience is large. Suitable for simple tasks. All players are.

The content we enter in the ChatGPT input box is the prompt.

9. ChatGPT is generative AI.

AI is divided into two types in terms of content production methods: analytical AI and generative AI.

Analytical AI is mainly used for analysis and classification. How much data you feed it, what kind of content it can analyze, it is limited to the data itself.

Generative AI creates new content that does not exist in the data based on learning and summarizing the distribution of data. Can generate text, pictures, code, speech synthesis, video and 3D models.

ChatGPT is a generative AI that is best at text and writing code.

10. Finally, from the perspective of knowledge acquisition, ChatGPT is a new generation of “knowledge representation and invocation method”.

In the early days, knowledge was stored in a structured manner in databases. We get it through SQL.

Later, with the birth of the Internet, more unstructured knowledge such as texts, pictures, and videos were stored on the Internet. We get it through search engines.

Now, knowledge is stored in the large model in the form of parameters. We directly invoke this knowledge using natural language.

The above is the detailed content of What is ChatGPT? What do G, P, and T stand for?. For more information, please follow other related articles on the PHP Chinese website!