Humanity has reached the upper limit of silicon computing architecture! AI is expected to consume 50% of global electricity supply by 2030

We have begun to experience the feeling of reaching the upper limit of silicon computing experience. In the next 10 years, there will be a serious computing power gap, and neither existing technology companies nor governments have been able to solve this problem.

Now, we are so accustomed to computing becoming cheaper and cheaper that we never doubt that one day we may not be able to afford it.

Now, Rodolfo Rosini, CEO of a startup, asks a question that shocks us: What if we are reaching the fundamental physical limits of the classical computing model, as if our economy relies on cheap computing? what to do?

The stagnation of large-scale computing

Now, due to a lack of technological innovation, the United States has reached a plateau.

Wright’s Law holds true in many industries—every time the manufacturing process is improved by about 20%, productivity will double.

In the field of technology, it manifests itself as Moore's Law.

In the 1960s, Intel co-founder Gordon Moore noticed that the number of transistors in integrated circuits seemed to have doubled year-on-year and proposed Moore's Law.

Since then, this law has become the basis of the contract between marketing and engineering, leveraging excess computing power and shrinking size to drive the construction of products in the computing stack.

The expectation was that computing power would improve exponentially over time with faster and cheaper processors.

However, the different forces that make up Moore’s Law have changed.

For decades, the driving force behind Moore’s Law was Dennard’s Law of Scaling. Transistor size and power consumption are simultaneously halved, doubling the amount of computation per unit of energy (the latter is also known as Koomey’s LawKoomey’s Law).

50 Years of Microprocessor Trend Data

In 2005, this scaling began to fail due to current leakage causing the chip to heat up, and the The problem is that the performance of chips with a single processing core has stagnated.

To maintain its computing growth trajectory, the chip industry has turned to multi-core architectures: multiple microprocessors "glued" together. While this may extend Moore's Law in terms of transistor density, it increases the complexity of the entire computing stack.

For certain types of computing tasks, such as machine learning or computer graphics, this brings performance improvements. But for many general-purpose computing tasks that are not well parallelized, multi-core architectures are powerless.

In short, the computing power for many tasks is no longer increasing exponentially.

Even in terms of the performance of multi-core supercomputers, judging from the TOP500 (ranking of the world's fastest supercomputers), there was an obvious turning point around 2010.

What are the impacts of this slowdown? The growing role computing plays in different industries shows that the impact is immediate and will only become more important as Moore's Law falters further.

To take two extreme examples: Increased computing power and reduced costs have led to a 49% increase in the productivity of oil exploration in the energy industry, and a 94% increase in protein folding predictions in the biotechnology industry.

This means that the impact of computing speed is not limited to the technology industry. Much of the economic growth of the past 50 years has been a second-order effect driven by Moore's Law. Without it, the world economy may stop growing.

Another prominent reason for the need for more computing power is the rise of artificial intelligence. Today, training a large language model (LLM) can cost millions of dollars and take weeks.

The future promised by machine learning cannot be realized without continued increase in number crunching and data expansion.

As the growing popularity of machine learning models in consumer technology heralds huge and potentially hyperbolic demands for computing in other industries, cheap processing is becoming the cornerstone of productivity.

The death of Moore's Law may bring about a great stagnation in computing. Compared with the multi-modal neural networks that may be required to achieve AGI, today's LLMs are still relatively small and easy to train. Future GPTs and their competitors will require particularly powerful high-performance computers to improve and even optimize.

Perhaps many people will feel doubtful. After all, the end of Moore's Law has been predicted many times. Why should it be now?

Historically, many of these predictions stem from engineering challenges. Human ingenuity has overcome these obstacles time and time again before.

The difference now is that instead of being faced with engineering and intelligence challenges, we are faced with constraints imposed by physics.

MIT Technology Review published an article on February 24 stating that we are not prepared for the end of Moore’s Law

Overheating causes inability to process

Computers work by processing information.

As they process information, some of it is discarded as the microprocessor merges computational branches or overwrites the registry. It's not free.

The laws of thermodynamics, which place strict limits on the efficiency of certain processes, apply to calculations just as they do to steam engines. This cost is called Landauer’s limit.

It is the tiny amount of heat emitted during each computational operation: approximately 10^-21 joules per bit.

Given that this heat is so small, the Landauer limit has long been considered negligible.

However, engineering capabilities have now evolved to the point where this energy scale can be achieved because the real-world limit is estimated to be 10-100 times larger than Landauer's bound due to other overheads such as current leakage. Chips have hundreds of billions of transistors that operate billions of times per second.

Add these numbers together, and perhaps Moore’s Law still has an order of magnitude left to grow before reaching the thermal barrier.

At that point, existing transistor architectures will be unable to further improve energy efficiency, and the heat generated will prevent transistors from being packed more tightly.

If we don’t figure this out, we won’t be able to see clearly how industry values ​​will change.

Microprocessors will be constrained and industry will compete for lower rewards for marginal energy efficiency.

The chip size will expand. Look at Nvidia's 4000-series GPU card: despite using a higher-density process, it's about the size of a small dog and packs a whopping 650W of power.

This prompted NVIDIA CEO Jen-Hsun Huang to declare in late 2022 that "Moore's Law is dead"—a statement that, while mostly true, has been denied by other semiconductor companies.

The IEEE releases a semiconductor roadmap every year. The latest assessment is that 2D shrinkage will be completed in 2028, and 3D shrinkage should be fully launched in 2031.

3D scaling (where chips are stacked on top of each other) is already common, but in computer memory, not in microprocessors.

This is because the heat dissipation of the memory is much lower; however, heat dissipation is complex in 3D architecture, so active memory cooling becomes important.

Memory with 256 layers is on the horizon, and is expected to reach the 1,000-layer mark by 2030.

Back to microprocessors, multi-gate device architectures that are becoming commercial standards (such as FinFETs and Gates-all-round) will continue to follow Moore's Law in the coming years.

However, due to inherent thermal problems, true vertical scaling was impossible after the 1930s.

In fact, current chipsets carefully monitor which parts of the processor are active at any time to avoid overheating even on a single plane.

2030 Crisis?

A century ago, American poet Robert Frost once asked: Will the world end in frost or fire?

If the answer is fire, it almost heralds the end of computing.

Or, just accept the fact that electricity use will increase and scale up microprocessor manufacturing.

For this purpose, humans have consumed a large portion of the earth's energy.

Perhaps another option is to simply accept increased power use and scale up microprocessor manufacturing. We already use a large portion of the Earth's energy supply for this purpose.

In Ireland, just 70 data centers consume 14% of the country’s energy. By the 2030s, it is expected that 30-50% of globally produced electricity will be used for computing and cooling.

(Interestingly, after the blog post was published on March 19, the author deleted this prediction. His explanation was that this was based on the worst-case scenario in the Nature paper. The extrapolation of the situation, which has now been deleted for the sake of clarity and precision of the argument)

And the current scale of energy production will lead to a slight increase in the cost of Moore's Law scale in the future.

A series of one-time optimization measures at the design (energy efficiency) and implementation levels (replacing old designs still in use with the latest technology) will allow developing economies such as India to catch up with the global overall productive forces.

After the end of Moore's Law, mankind will run out of energy before the manufacturing of microprocessor chips reaches its limit, and the pace of decline in computing costs will stagnate.

Although quantum computing is touted as an effective way to surpass Moore's Law, there are too many unknowns in it. It is still decades away from commercial development, and it will not be useful for at least the next 20 to 30 years. .

Clearly, there will be a serious computing power gap in the next 10 years, and no existing technology companies, investors, or government agencies will be able to solve it.

The collision between Moore's Law and Landauer's limit has been going on for decades, and it can be said to be one of the most significant and critical events in the 2030s.

But now, it seems that not many people know about this matter.

The above is the detailed content of Humanity has reached the upper limit of silicon computing architecture! AI is expected to consume 50% of global electricity supply by 2030. For more information, please follow other related articles on the PHP Chinese website!