The self-organizing neural network that "grows" like a biological network and has "structural plasticity" is here

Biological Neural NetworksAn important feature is a high degree of plasticity, which makes natural organisms have excellent adaptability, and this ability affects the synaptic strength and topology of the nervous system.

However, artificial neural networks are mainly designed as static, fully connected structures, which can be very fragile in the face of changing environments and new inputs. Although researchers have conducted extensive research into online learning and meta-learning, current state-of-the-art neural network systems still use offline learning because this is simpler when combined with backpropagation.

So, can artificial neural networks also have properties similar to high plasticity?

A research team from the University of Information Technology in Copenhagen proposed a self-organizing neural network - LNDP, which can achieve synaptic and structural plasticity in an activity- and reward-dependent manner.

• Paper link: https://arxiv.org/pdf/2406.09787

• Project link: https://github.com/erwanplantec/LNDP

Research Introduction

In 2023, Najarro et al proposed the Neurodevelopmental Program (NDP) model. But the NDP is limited in time to the pre-environmental stage. Therefore, a research team from the Information Technology University of Copenhagen addressed this limitation by extending the NDP framework.

Specifically, the research team proposed a mechanism that can achieve plasticity and structural changes during the life cycle of an agent - LNDP (Lifelong Neural Developmental programs). This mechanism is implemented by performing local computations, relying on the local activity of each neuron in the artificial neural network and the global reward function of the environment. LNDP makes artificial neural networks plastic and bridges the plasticity rules of indirect developmental encoding and meta-learning.

LNDP consists of a set of parameterized components designed to define neural and synaptic dynamics and make artificial neural networks structurally plastic (i.e. synapses can be added or removed dynamically).

Inspired by biological spontaneous activity (SA), the research team further expanded the system and introduced a mechanism that can realize the development of pre-experience, using the simple data of sensory neurons. Learning stochastic processes model SA, which makes some components reusable.

The research team proposed an LNDP instance based on the Graph Transformer layer (Dwivedi and Bresson, 2021) and optimized LNDP using the covariance matrix adaptive evolution strategy (CMA-ES) on a set of reinforcement learning tasks.

Specifically, this study used three classic control tasks (Cartpole, Acrobot, Pendulum) and a collection task (Foraging) with non-stationary dynamics, which require the agent to have life cycle adaptability.

In summary, the research team demonstrated that starting from a randomly connected (or empty) neural network, LNDP self-organizes to form a functional network in an activity- and experience-dependent manner to effectively solve control tasks.

The study also shows that structural plasticity can improve outcomes in environments that require rapid adaptation or have non-stationary dynamics that require continuous adaptation. Furthermore, this study demonstrates the effectiveness of developmental stages driven by pre-environmental spontaneous activity in network self-organization into functional units.

Experimental results

The research team tested the differences between SP models (models with structural plasticity) and non-SP models (models without structural plasticity) on all tasks, and the results are shown in Figure 2 below.

비정상 역학을 사용한 수집 작업(Foraging)에서 연구팀은 SP 모델이 항상 비SP 모델보다 더 높은 평균 적합도를 달성했으며 두 모델은 비슷한 최대 적합도에 도달했음을 발견했습니다. 이는 SP가 고정되지 않은 상황에서 더 나은 적응성을 갖는다는 것을 보여줍니다.

CartPole 환경에서는 특히 SA가 없는 모델은 처음부터 좋은 성과를 내기 어려운 반면, SA가 있는 모델은 처음부터 문제를 해결하는 고유한 능력을 보여줍니다. 이는 보상에 의존하지 않고 자체 조직화되는 방식으로 목표 기능 네트워크를 달성하는 모델의 능력을 보여줍니다.

자세한 연구 내용은 원문을 참고해주세요.

The above is the detailed content of The self-organizing neural network that "grows" like a biological network and has "structural plasticity" is here. For more information, please follow other related articles on the PHP Chinese website!