What to do if Moore's Law fails? Neuromorphic Computing Expert: Turning the Focus to Dendritic Learning

In 1965, Gordon Moore summarized a rule of thumb: the number of transistors that can be accommodated on an integrated circuit will double approximately every 18 to 24 months. In other words, processor performance doubles approximately every two years.

This rule of thumb is called "Moore's Law". In the following forty years, the semiconductor chip manufacturing process has indeed progressed at a dizzying rate. The speed is doubled. However, in recent years, the doubling effect of Moore's Law has been slowing down, and some even predict that it will expire in the near future.

The industry has proposed various solutions to deal with this development bottleneck. Kwabena Boahen, a neuromorphic engineer from Stanford University, recently proposed a new idea: artificial neurons should imitate the functions of biological neurons. Dendrites, not synapses. The research paper was published in Nature.

Paper address: https://www.nature.com/articles/s41586-022-05340-6

Currently, neuromorphic computing aims to enable artificial intelligence (AI) by mimicking the mechanisms of neurons and synapses that make up the human brain. Artificial neural networks repeatedly adjust the synapses connecting neurons to modify each synapse's "weight," or the strength of one neuron's influence on another. The neural network then determines whether the resulting behavioral patterns are better at finding them. solution. Over time, the system discovers which modes are best for calculating results and adopts those modes as the default.

Neural networks usually contain many layers of neurons. For example, GPT-3 has 175 billion weights, connections equivalent to 8.3 million neurons, and a depth of 384 layers. As neural networks continue to increase in size and functionality, they become increasingly expensive and energy-intensive. Taking GPT-3 as an example, OpenAI spent $4.6 million to run 9,200 GPUs for two weeks to train this large model. Kwabena Boahen said: "The energy consumed by GPT-3 during training is converted into carbon emissions equivalent to 1,300 cars."

This is why Boahen proposed that the next step for neural networks should be to try graph learning important reasons. Mimicking dendrites in neural networks will increase the amount of information conveyed in transmitted signals, allowing AI systems to no longer require megawatts of power in the GPU cloud and run on mobile devices such as mobile phones.

Dendrites can branch massively, allowing one neuron to connect with many other neurons. Studies have found that the order in which a dendrite receives signals from its branches determines the strength of its response.

The computational model of dendrites proposed by Boahen only makes decisions when it receives a precise sequence of signals from the neuron. reaction. This means that each dendrite can encode data, not just simple electrical signals like 0/1. The base system will become more powerful depending on the number of connections it has and the length of the signal sequence it receives.

In terms of actual construction, Boahen proposed using ferroelectric FETs (FeFETs) to simulate dendrites. A 1.5-micron-long FeFET with 5 gates can simulate 5 synapses. of 15 micron long dendrites. A version of this build might be implemented in a "3D chip," Boahen said.

Interested readers can read the original text of the paper to learn more about the research details.

Reference link: https://spectrum.ieee.org/dendrocentric-learning​

The above is the detailed content of What to do if Moore's Law fails? Neuromorphic Computing Expert: Turning the Focus to Dendritic Learning. For more information, please follow other related articles on the PHP Chinese website!