ChatGPT 'Nemesis': Use AI to recognize AI-generated text, and English paper reading notes can be detected

The emergence of ChatGPT has allowed many people to see the dawn of a big job at the end of the deadline (manual dog head).

Whether it is an English paper or reading notes, as long as it is within the knowledge scope of ChatGPT, you can ask it to help you complete it, and the written content will be well-founded.

However, have you ever thought that your teacher is also planning to use something like an "AI text detector" to prevent you from cheating?

Enter a seemingly flawless note like this, and after some testing, the probability that this text is "written by AI" (Fake) is 99.98%!

△The text is generated by ChatGPT

Try another math paper? The output of ChatGPT seems to have no problem, but it is still accurately recognized by it:

△The text is generated by ChatGPT

This is not relying on blindness or Guess, after all, the other party is also an AI, and a well-trained AI.

After seeing this, some netizens joked: Use magic to defeat magic?

Use things written by AI to train new AI

This AI detector is called GPT-2 Output Detector, which is a joint venture between OpenAI and Harvard University and other universities. Created together with the organization. (Yes, OpenAI makes it in-house)

You can more accurately identify text generated by AI by entering more than 50 characters (tokens).

But even a model that specializes in detecting GPT-2 is equally effective in detecting text generated by other AIs.

The authors first released a data set of "GPT-2 generated content" and WebText (specially taken from Reddit, a foreign post bar), allowing AI to understand the difference between "AI language" and "human speech" difference.

Subsequently, this data set was used to fine-tune the RoBERTa model, and the AI ​​detector was obtained.

RoBERTa (Robustly Optimized BERT approach) is an improved version of BERT. The original BERT used a 13GB dataset, but RoBERTa used a 160GB dataset containing 63 million English news items.

Among them, human speech is always recognized as True, and AI-generated content is always recognized as Fake.

For example, this is a piece of content copied from Medium’s English blog. Judging from the recognition results, it is obvious that the author wrote it himself (manual dog head):

△Text source Medium@Megan Ng

Of course, this detection The device is not 100% accurate either.

The larger the number of AI model parameters, the harder it is for the generated content to be identified. For example, a model with 124 million parameters has a higher probability of being "captured" than a model with 1.5 billion parameters.

At the same time, the higher the randomness of the model generation results, the lower the probability of AI-generated content being detected.

But even if the model is adjusted to generate the highest randomness (Temperature=1, the closer to 0, the lower the randomness), the probability of being detected by the 124 million parameter model is still 88%, and the 1.5 billion parameter model is detected The probability of detection is still 74%.

This is a model released by OpenAI two years ago. At that time, the content generated by GPT-2 was "accurate".

Now facing the upgraded version of ChatGPT, the effect of detecting English-generated content can still be achieved.

But when it comes to Chinese generated by ChatGPT, its recognition ability is not that good. For example, let ChatGPT write a composition:

##The AI ​​detector gives a probability of 99.96% that it was written by a human...

Of course, having said that, ChatGPT can also detect the text it generates.

Therefore, it is not ruled out that the teacher will hand your homework directly to ChatGPT for identification:

##One More Thing

Worth mentioning Yes, ChatGPT stated that it cannot access the Internet to search for information.

Obviously, it is not aware of the existence of the GPT-2 Output Detector AI detector:

So, as netizens said, can ChatGPT generate a piece of content that is “not detected by the AI ​​detector”?

Unfortunately, I can’t:

So I’d better write the big homework by myself...

Reference Link:[1]​​https://weibo.com/1402400261/Mj7QtwRoH[2]https://github.com/openai/gpt-2-output-dataset/tree/master/detector[3] https://chat.openai.com​​/

[4]https://medium.com/user-experience-design-1/how-chatgpt-is-blowing-google- out-of-the-water-a-ux-breakdown-784340c25d57

The above is the detailed content of ChatGPT 'Nemesis': Use AI to recognize AI-generated text, and English paper reading notes can be detected. For more information, please follow other related articles on the PHP Chinese website!