


Xi Xiaoyao Technology Talk Original
Author | PythonChatGPT, as an intelligent human-machine conversation application, quickly became popular all over the world after its launch. In just one month, its number of users has exceeded 100 million. People also use ChatGPT to test many exam items, such as SAT, AP, GRE, etc. However, what would it be like if ChatGPT were allowed to participate in China’s college entrance examination? Will he be biased? Can ordinary people like us pass ChatGPT? Let’s take a look at the comments brought to you by students from Fudan University and East China Normal University.
Paper title:
Evaluating the Performance of Large Language Models on GAOKAO Benchmark
Paper link:https://www.php.cn/link/de66dbc30377a1f7908db45298324aa3
How to use ChatGPT to answer college entrance examination questions?
This paper uses a zero-supervised prompt method to convert test questions into ChatGPT input, as shown in the figure below. Different inquiry methods are designed for different subjects and question types. For math questions, convert the formula into latex input.
College Entrance Examination Data Set
The test in this article uses the national A paper and the national B paper from 2010 to 2022, a total of 13 years. Each set of test papers contains 10 subjects Subjects include Chinese, Mathematics, English, Physics, Chemistry, Life, History, and Geography. Mathematics is divided into science mathematics and liberal arts mathematics.
The data set contains a total of 2811 test questions. The specific question types will not be discussed here. I believe readers are very familiar with the college entrance examination questions.
During the evaluation, high school teachers from Caoyang No. 2 Middle School in Shanghai were hired to mark the subjective questions.
Experiment and Analysis
The scores obtained by ChatGPT in the college entrance examination over the past years are shown in the figure below. Since each subject is normalized to 100 points when calculating scores, this score cannot be directly compared with your and my college entrance examination scores. But it can also be seen that this score is not ideal, and it is estimated that neither Fudan nor East China Normal University will be admitted. Why is this?
The above picture shows the performance of ChatGPT in various disciplines, as well as subjective and objective topics. Blue is objective questions and yellow is subjective questions. The analysis found that ChatGPT performed better on objective questions, especially English reading comprehension, multiple choice, and cloze, achieving 88.3%, 78.1%, and 73.8% accuracy respectively. But even for objective questions, the accuracy rate in science and mathematics is less than 40%. Mathematics is really difficult~
ChatGPT’s performance on subjective questions is poor. In the subjects of physics, chemistry, biology and mathematics, the performance of subjective questions is obviously not as good as objective questions. The scores on objective questions in science subjects are also poor. Maybe ChatGPT is biased towards liberal arts subjects? According to the reviewers' comments, the main shortcomings of ChatGPT are: 1. The complex equations in mathematical problems are difficult to solve correctly, and wrong formulas are used in the problem solving process. 2. Inadequate ability to understand and summarize when reading longer materials.
Summary
ChatGPT may not have used Chinese college entrance examination question data during the training process, so its performance is not affected by data leaks and has high credibility.
Observation results show that compared with foreign exams, ChatGPT’s performance on Chinese college entrance examination questions is slightly inferior. Therefore, domestic students do not need to worry too much about not being able to pass the ChatGPT exam for the time being. However, the long text summarization capabilities mentioned in the article have been significantly improved in GPT4-32K, and the domestic large model has also been further optimized on Chinese data. Therefore, we can expect more impressive performance on large-model college entrance examination questions in the future.
In addition, the idea of using ChatGPT to solve college entrance examination questions may be able to answer the debate among netizens about which province’s test questions are more difficult?
The above is the detailed content of When ChatGPT took part in China's college entrance examination and fed it the national A and B papers, it turned out to be seriously biased!. For more information, please follow other related articles on the PHP Chinese website!

The Model Context Protocol (MCP): A Universal Connector for AI and Data We're all familiar with AI's role in daily coding. Replit, GitHub Copilot, Black Box AI, and Cursor IDE are just a few examples of how AI streamlines our workflows. But imagine

Microsoft's OmniParser V2 and OmniTool: Revolutionizing GUI Automation with AI Imagine AI that not only understands but also interacts with your Windows 11 interface like a seasoned professional. Microsoft's OmniParser V2 and OmniTool make this a re

Revolutionizing App Development: A Deep Dive into Replit Agent Tired of wrestling with complex development environments and obscure configuration files? Replit Agent aims to simplify the process of transforming ideas into functional apps. This AI-p

Vibe coding is reshaping the world of software development by letting us create applications using natural language instead of endless lines of code. Inspired by visionaries like Andrej Karpathy, this innovative approach lets dev

This blog post shares my experience testing Runway ML's new Act-One animation tool, covering both its web interface and Python API. While promising, my results were less impressive than expected. Want to explore Generative AI? Learn to use LLMs in P

YOLO (You Only Look Once) has been a leading real-time object detection framework, with each iteration improving upon the previous versions. The latest version YOLO v12 introduces advancements that significantly enhance accuracy

February 2025 has been yet another game-changing month for generative AI, bringing us some of the most anticipated model upgrades and groundbreaking new features. From xAI’s Grok 3 and Anthropic’s Claude 3.7 Sonnet, to OpenAI’s G

The $500 billion Stargate AI project, backed by tech giants like OpenAI, SoftBank, Oracle, and Nvidia, and supported by the U.S. government, aims to solidify American AI leadership. This ambitious undertaking promises a future shaped by AI advanceme


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

mPDF
mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),

SublimeText3 Chinese version
Chinese version, very easy to use

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft
