MIT's new materials create "Artificial Synapse 2.0" to simulate deep learning training and speed up 1 million times!

In recent years, as scientists continue to push the boundaries of machine learning, the time, energy, and money required to train increasingly complex neural network models have grown rapidly. "The model can be built, but the training is too slow" has become a headache that troubles more and more researchers.

Recently, a new field of artificial intelligence called "analog deep learning" promises faster computing with less energy. Programmable resistors are a key part of analog deep learning, just as transistors are core elements of digital processors.

By repeatedly arranging programmable resistors in complex layers, researchers can create a simulated network of artificial neurons and synapses that perform calculations like a digital neural network. This network can be trained to implement complex artificial intelligence tasks such as image recognition and natural language processing.

The calculation speed of the simulated neural network depends largely on the transmission speed of the "artificial synapse". This is what a team from MIT is trying to solve. They have previously developed an artificial simulated synapse, and what they need to do now is to develop a new material that surpasses the original old version.

This time, they used a practical inorganic material in the manufacturing process to allow the aforementioned programmable resistor to operate a million times faster than previous versions. , and also achieved about 1 million times faster than the synapses in the human brain. In addition, this material makes resistors extremely energy efficient. Unlike those used in earlier versions of the device, the new material is compatible with silicon manufacturing technology. This change makes it possible to fabricate devices at the nanometer scale and may pave the way for integration into commercial computing hardware for deep learning applications. This research paper has been published in Science.

Paper link: https://www.science.org/doi/pdf/10.1126/science.abp8064

"With this key discovery, we are able to Coupled with MIT.nano's powerful nanofabrication technology, we have been able to put these pieces together and demonstrate that these devices are inherently very fast and can operate at reasonable voltages." "The device works by The smallest ions - protons - are electrochemically inserted into the insulating oxide to modulate its electronic conductivity. Because the devices we use are so thin, we can accelerate the movement of this ion and push these ion devices through the use of strong electric fields. Towards nanosecond operations," explained Bilge Yildiz, corresponding author of the paper and Breene M. Kerr Professor in the Department of Nuclear Science and Engineering and the Department of Materials Science and Engineering. Ju Li, corresponding author of the paper and a professor in the Department of Nuclear Science and Engineering and a professor in the Department of Materials Science and Engineering at Battelle Energy Alliance, said:

"Action potentials in biological cells rise and fall on a millisecond scale. , because a voltage difference of about 0.1 volts is governed by the stability of water." Here we apply voltages up to 10 volts on a special nanometer-thick solid glass film that can conduct protons without permanent damage it. Also, the stronger the electric field, the faster the ionization device will operate. Co-authors of the paper also include Frances M. Ross, the Ellen Swallow Richards Professor in the Department of Materials Science and Engineering; postdocs Nicolas Emond and Baoming Wang; and EECS graduate student Difei Zhang.

These programmable resistors greatly increase the speed of training neural networks while greatly reducing the cost and energy consumption of training. This could help scientists develop deep learning models more quickly for applications such as self-driving cars, fraud detection, or medical image analysis. "Once you have an analog processor, you no longer have to train the network that other people are working on. You can train a network with unprecedentedly high complexity that no one else can afford. In other words, it's not just Not just a fast car, but a plane." Said Murat Onen, the first author of the paper and a postdoctoral fellow at MIT.

1 million times accelerated deep learning

There are two main reasons why analog deep learning is faster and more energy-efficient than its digital counterpart.

First of all, the calculations are performed in memory, so huge data loads are not transferred back and forth from memory to the processor. Analog processors also operate in parallel. If the matrix grows, the analog processor does not need more time to complete the new operation because all calculations are performed simultaneously. A key element of MIT's new analog processor technology is called a proton-programmable resistor. These resistors are measured in nanometers and are arranged in an array, like a checkerboard.

Nanosecond Proton Programmable Resistor

In order to develop an ultra-high speed, high energy efficiency For programmable proton resistance, the researchers looked at different electrolyte materials. While other devices use organic compounds, Onen focuses on inorganic phosphosilicate glass (PSG). PSG is basically silica, a powdered desiccant material used to remove moisture.

The researchers studied it as a proton conductor for fuel cells under humidified conditions. It is also the best-known oxide in silicon processing. To make PSG, a small amount of phosphorus is added to the silicon, giving it special properties of proton conduction. Onen hypothesized that the optimized PSG could have high proton conductivity at room temperature without the need for water, which would make it an ideal solid electrolyte.

Amazing Speed

PSG is capable of ultrafast motion of protons because it contains a large number of nanometer-sized pores whose surface provides paths for protons to diffuse.

It can also withstand very strong pulsed electric fields.

#This is crucial because applying more voltage to the device can make the protons move extremely fast," explains Onen. "This speed is quite surprising," Onen said. "Normally, we wouldn't use such strong magnetic fields on devices to avoid them turning to ash. But instead, the protons ended up zipping through the pile of devices at tremendous speeds. And it's still a million times faster than what we were doing before. Also, because protons are small and have low mass, this motion doesn't destroy anything. It's almost like teleportation."

Because protons do not damage the material, resistors can operate for millions of cycles without failure.

Ultrafast and energy-efficient modulation properties of proton programmable resistors

## This new electrolyte enables programmable proton resistors that are 1 million times faster than previous devices and can operate efficiently at room temperature, which is important for their incorporation into computing hardware.

Due to the insulating properties of PSG, when protons move, almost no current flows through the material. "This makes the device very energy efficient," Onen added. "Now that they have demonstrated the effectiveness of these programmable resistors, the researchers plan to develop them," del Alamo said. Redesign for high-volume production. They can then study the properties of the resistor array and scale it up to embed it into a system."

Yildiz added: "These Another exciting direction that Ionic devices can enable is energy-efficient hardware to simulate neural circuits and synaptic plasticity rules derived from neuroscience, beyond simulating deep neural networks. We have already begun such collaborations with neuroscience, getting Supported by MIT Quest for Intelligence.”

Our collaboration is critical to future innovation. The road ahead remains full of challenges Challenging, but also very exciting!"

William Chueh, associate professor of materials science and engineering at Stanford University, said: "Intercalation reactions such as those found in lithium-ion batteries have been used in memory devices. have been extensively explored. This work shows that proton-based memory devices offer impressive and surprising switching speeds and durability."

######But William Chueh did not did not participate in this study. ######

The above is the detailed content of MIT's new materials create "Artificial Synapse 2.0" to simulate deep learning training and speed up 1 million times!. For more information, please follow other related articles on the PHP Chinese website!