


AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black box
When AI does math problems, the real thinking is actually "mental arithmetic" secretly?
New research by a New York University team found that even if AI is not allowed to write steps and is replaced with meaningless "...", its performance on some complex tasks can be greatly improved!
First author Jacab Pfau said: As long as you spend computing power to generate additional tokens, you can bring advantages. It doesn’t matter what token you choose.
Picture
For example, let Llama 34M answer a simple question: How many of the first 6 digits of the natural constant e are greater than 5 ?
The AI's direct answer is equivalent to making trouble. It only counts the first 6 digits and actually counts 7.
Let AI write out the steps to verify each number, and you can get the correct answer.
Let AI hide the steps and replace them with a lot of "...", and you can still get the correct answer!
Picture
This paper sparked a lot of discussion as soon as it was released, and was evaluated as "the most metaphysical AI paper I have ever seen."
Picture
So, young people like to say more meaningless words such as "um...", "like...", is it okay? Strengthen reasoning skills?
Picture
From thinking "step by step" to thinking "little by little"
In fact, the New York University team The research starts from the Chain-of-Thought (CoT).
That’s the famous prompt “Let’s think step by step”.
Picture
In the past, it was found that using CoT inference can significantly improve the performance of large models on various benchmarks.
It’s unclear whether this performance improvement comes from imitating humans by breaking tasks into easier-to-solve steps, or whether it is a by-product of the extra calculations.
In order to verify this problem, the team designed two special tasks and corresponding synthetic data sets: 3SUM and 2SUM-Transform.
3SUM requires finding three numbers from a given set of number sequences so that the sum of the three numbers satisfies certain conditions, such as dividing by 10 with a remainder of 0.
Picture
The computational complexity of this task is O(n3), and the standard Transformer uses the input of the upper layer and the activation of the next layer Only secondary dependencies can occur between them.
That is to say, when n is large enough and the sequence is long enough, the 3SUM task exceeds the expression ability of Transformer.
In the training data set, "..." with the same length as the human reasoning steps is filled between the question and the answer. That is, the AI has not seen how humans disassemble the problem during training.
Picture
In the experiment, the performance of Llama 34M that does not output the padding token "..." decreases as the sequence length increases, while the output When filling the token, 100% accuracy can be guaranteed until the length is 14.
Picture
2SUM-Transform only needs to determine whether the sum of two numbers meets the requirements, which is within the expressive capabilities of Transformer.
But at the end of the question, a step is added to "randomly replace each number of the input sequence" to prevent the model from directly calculating on the input token.
The results show that using padding tokens can increase the accuracy from 78.7% to 93.6%.
picture
In addition to the final accuracy, the author also studied the hidden layer representation of the filled token. Experiments show that by freezing the parameters of the previous layers and only fine-tuning the last Attention layer, the prediction accuracy increases as the number of available filling tokens increases.
This confirms that the hidden layer representation of the populated token does contain implicit computation related to downstream tasks.
Picture
Has AI learned to hide its thoughts?
Some netizens wonder, is this paper saying that the "thinking chain" method is actually fake? The prompt word project that I have been studying for so long has been in vain.
Picture
The team stated that theoretically the role of filling tokens is limited to the scope of TC0 complexity problems.
TC0 is a computational problem that can be solved by a fixed-depth circuit, in which each layer of the circuit can be processed in parallel and can be quickly solved by a few layers of logic gates (such as AND, OR and NOT gates) , which is also the upper limit of computational complexity that Transformer can handle in single forward propagation.
And a long enough thinking chain can extend the expression ability of Transformer beyond TC0.
And it is not easy for a large model to learn to use padding tokens, and specific intensive supervision needs to be provided to converge.
That said, existing large models are unlikely to benefit directly from the padding token method.
But this is not an inherent limitation of current architectures; if provided with sufficient demonstrations in the training data, they should be able to obtain similar benefits from padding symbols.
This research also raises a worrying issue: large models have the ability to perform secret calculations that cannot be monitored, posing new challenges to the explainability and controllability of AI.
In other words, AI can reason on its own in a form invisible to people without relying on human experience.
This is both exciting and terrifying.
Picture
Finally, some netizens jokingly suggested that Llama 3 first generate 1 quadrillion dots, so that the weight of AGI can be obtained (dog head) .
Picture
Paper:https://www.php.cn/link/36157dc9be261fec78aeee1a94158c26
Reference Link:
[1]https://www.php.cn/link/e350113047e82ceecb455c33c21ef32a[2]https://www.php.cn/link/872de53a900f3250ae5649ea19e5c381
The above is the detailed content of AI learns to hide its thinking and reason secretly! Solving complex tasks without relying on human experience is more black box. For more information, please follow other related articles on the PHP Chinese website!

In John Rawls' seminal 1971 book The Theory of Justice, he proposed a thought experiment that we should take as the core of today's AI design and use decision-making: the veil of ignorance. This philosophy provides a simple tool for understanding equity and also provides a blueprint for leaders to use this understanding to design and implement AI equitably. Imagine that you are making rules for a new society. But there is a premise: you don’t know in advance what role you will play in this society. You may end up being rich or poor, healthy or disabled, belonging to a majority or marginal minority. Operating under this "veil of ignorance" prevents rule makers from making decisions that benefit themselves. On the contrary, people will be more motivated to formulate public

Numerous companies specialize in robotic process automation (RPA), offering bots to automate repetitive tasks—UiPath, Automation Anywhere, Blue Prism, and others. Meanwhile, process mining, orchestration, and intelligent document processing speciali

The future of AI is moving beyond simple word prediction and conversational simulation; AI agents are emerging, capable of independent action and task completion. This shift is already evident in tools like Anthropic's Claude. AI Agents: Research a

Rapid technological advancements necessitate a forward-looking perspective on the future of work. What happens when AI transcends mere productivity enhancement and begins shaping our societal structures? Topher McDougal's upcoming book, Gaia Wakes:

Product classification, often involving complex codes like "HS 8471.30" from systems such as the Harmonized System (HS), is crucial for international trade and domestic sales. These codes ensure correct tax application, impacting every inv

The future of energy consumption in data centers and climate technology investment This article explores the surge in energy consumption in AI-driven data centers and its impact on climate change, and analyzes innovative solutions and policy recommendations to address this challenge. Challenges of energy demand: Large and ultra-large-scale data centers consume huge power, comparable to the sum of hundreds of thousands of ordinary North American families, and emerging AI ultra-large-scale centers consume dozens of times more power than this. In the first eight months of 2024, Microsoft, Meta, Google and Amazon have invested approximately US$125 billion in the construction and operation of AI data centers (JP Morgan, 2024) (Table 1). Growing energy demand is both a challenge and an opportunity. According to Canary Media, the looming electricity

Generative AI is revolutionizing film and television production. Luma's Ray 2 model, as well as Runway's Gen-4, OpenAI's Sora, Google's Veo and other new models, are improving the quality of generated videos at an unprecedented speed. These models can easily create complex special effects and realistic scenes, even short video clips and camera-perceived motion effects have been achieved. While the manipulation and consistency of these tools still need to be improved, the speed of progress is amazing. Generative video is becoming an independent medium. Some models are good at animation production, while others are good at live-action images. It is worth noting that Adobe's Firefly and Moonvalley's Ma

ChatGPT user experience declines: is it a model degradation or user expectations? Recently, a large number of ChatGPT paid users have complained about their performance degradation, which has attracted widespread attention. Users reported slower responses to models, shorter answers, lack of help, and even more hallucinations. Some users expressed dissatisfaction on social media, pointing out that ChatGPT has become “too flattering” and tends to verify user views rather than provide critical feedback. This not only affects the user experience, but also brings actual losses to corporate customers, such as reduced productivity and waste of computing resources. Evidence of performance degradation Many users have reported significant degradation in ChatGPT performance, especially in older models such as GPT-4 (which will soon be discontinued from service at the end of this month). this


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Atom editor mac version download
The most popular open source editor

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

Zend Studio 13.0.1
Powerful PHP integrated development environment

SublimeText3 English version
Recommended: Win version, supports code prompts!

Notepad++7.3.1
Easy-to-use and free code editor
