


Xi Xiaoyao Technology Talk Original
Author | PythonChatGPT, as an intelligent human-machine conversation application, quickly became popular all over the world after its launch. In just one month, its number of users has exceeded 100 million. People also use ChatGPT to test many exam items, such as SAT, AP, GRE, etc. However, what would it be like if ChatGPT were allowed to participate in China’s college entrance examination? Will he be biased? Can ordinary people like us pass ChatGPT? Let’s take a look at the comments brought to you by students from Fudan University and East China Normal University.
Paper title:
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Paper link:https://www.php.cn/link/de66dbc30377a1f7908db45298324aa3
How to use ChatGPT to answer college entrance examination questions?
This paper uses a zero-supervised prompt method to convert test questions into ChatGPT input, as shown in the figure below. Different inquiry methods are designed for different subjects and question types. For math questions, convert the formula into latex input.
College Entrance Examination Data Set
The test in this article uses the national A paper and the national B paper from 2010 to 2022, a total of 13 years. Each set of test papers contains 10 subjects Subjects include Chinese, Mathematics, English, Physics, Chemistry, Life, History, and Geography. Mathematics is divided into science mathematics and liberal arts mathematics.
The data set contains a total of 2811 test questions. The specific question types will not be discussed here. I believe readers are very familiar with the college entrance examination questions.
During the evaluation, high school teachers from Caoyang No. 2 Middle School in Shanghai were hired to mark the subjective questions.
Experiment and Analysis
The scores obtained by ChatGPT in the college entrance examination over the past years are shown in the figure below. Since each subject is normalized to 100 points when calculating scores, this score cannot be directly compared with your and my college entrance examination scores. But it can also be seen that this score is not ideal, and it is estimated that neither Fudan nor East China Normal University will be admitted. Why is this?
The above picture shows the performance of ChatGPT in various disciplines, as well as subjective and objective topics. Blue is objective questions and yellow is subjective questions. The analysis found that ChatGPT performed better on objective questions, especially English reading comprehension, multiple choice, and cloze, achieving 88.3%, 78.1%, and 73.8% accuracy respectively. But even for objective questions, the accuracy rate in science and mathematics is less than 40%. Mathematics is really difficult~
ChatGPT’s performance on subjective questions is poor. In the subjects of physics, chemistry, biology and mathematics, the performance of subjective questions is obviously not as good as objective questions. The scores on objective questions in science subjects are also poor. Maybe ChatGPT is biased towards liberal arts subjects? According to the reviewers' comments, the main shortcomings of ChatGPT are: 1. The complex equations in mathematical problems are difficult to solve correctly, and wrong formulas are used in the problem solving process. 2. Inadequate ability to understand and summarize when reading longer materials.
Summary
ChatGPT may not have used Chinese college entrance examination question data during the training process, so its performance is not affected by data leaks and has high credibility.
Observation results show that compared with foreign exams, ChatGPT’s performance on Chinese college entrance examination questions is slightly inferior. Therefore, domestic students do not need to worry too much about not being able to pass the ChatGPT exam for the time being. However, the long text summarization capabilities mentioned in the article have been significantly improved in GPT4-32K, and the domestic large model has also been further optimized on Chinese data. Therefore, we can expect more impressive performance on large-model college entrance examination questions in the future.
In addition, the idea of using ChatGPT to solve college entrance examination questions may be able to answer the debate among netizens about which province’s test questions are more difficult?
The above is the detailed content of When ChatGPT took part in China's college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!. For more information, please follow other related articles on the PHP Chinese website!

Vibe coding is reshaping the world of software development by letting us create applications using natural language instead of endless lines of code. Inspired by visionaries like Andrej Karpathy, this innovative approach lets dev

DALL-E 3: A Generative AI Image Creation Tool Generative AI is revolutionizing content creation, and DALL-E 3, OpenAI's latest image generation model, is at the forefront. Released in October 2023, it builds upon its predecessors, DALL-E and DALL-E 2

February 2025 has been yet another game-changing month for generative AI, bringing us some of the most anticipated model upgrades and groundbreaking new features. From xAI’s Grok 3 and Anthropic’s Claude 3.7 Sonnet, to OpenAI’s G

YOLO (You Only Look Once) has been a leading real-time object detection framework, with each iteration improving upon the previous versions. The latest version YOLO v12 introduces advancements that significantly enhance accuracy

The $500 billion Stargate AI project, backed by tech giants like OpenAI, SoftBank, Oracle, and Nvidia, and supported by the U.S. government, aims to solidify American AI leadership. This ambitious undertaking promises a future shaped by AI advanceme

Google's Veo 2 and OpenAI's Sora: Which AI video generator reigns supreme? Both platforms generate impressive AI videos, but their strengths lie in different areas. This comparison, using various prompts, reveals which tool best suits your needs. T

Google DeepMind's GenCast: A Revolutionary AI for Weather Forecasting Weather forecasting has undergone a dramatic transformation, moving from rudimentary observations to sophisticated AI-powered predictions. Google DeepMind's GenCast, a groundbreak

The article discusses AI models surpassing ChatGPT, like LaMDA, LLaMA, and Grok, highlighting their advantages in accuracy, understanding, and industry impact.(159 characters)


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

SublimeText3 English version
Recommended: Win version, supports code prompts!

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

SublimeText3 Linux new version
SublimeText3 Linux latest version

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.
