A review of sample pool calculations in machine learning

Reservoir computing (RC) is a computing framework that uses recursive neural networks. Unlike traditional neural networks, it only updates some parameters and selects them randomly. and fix other parameters.

The reserve pool is a fixed nonlinear system that maps input signals to a higher-dimensional computational space through its dynamics. The reservoir can be regarded as a black box. After the input signal is fed back to the reservoir, a simple readout mechanism is trained to read the state of the reservoir and map it to the required output.

Since the reservoir dynamics are fixed, training only occurs during the readout phase.

Traditional reserve pool calculations need to meet two conditions: they are composed of independent nonlinear units and can store information.

Reserve pool calculation is essentially a method used to make machine learning algorithms run faster.

The "Reservoir" in the term refers to the power system. A dynamical system is represented by a mathematical function that explains how points in space change over time. Knowing this allows you to predict the location of the point in space.

The reserve pool consists of several randomly connected circular connection units. The reserve pool calculation utilizes a recurrent neural network. Instead of updating all parameters of the network, it only updates a few parameters and randomly Keep other parameters unchanged after selection.

The framework of reserve pool calculation is similar to the framework of recursive neural network. Echo state network, liquid state machine and other recursive neural network models constitute the basic framework of reserve pool calculation. It is this Designed to make reserve pool computation quite efficient on tasks that deal with temporal or sequential data.

The purpose of reserve pool calculation

The reserve pool calculation is to sequentially transform the nonlinear input into a high-dimensional space so that it can be Learning algorithms read out the characteristics of the input in an efficient manner. In addition to using recurrent neural networks, other dynamical systems can also be used as reserve pools. The goal of reserve pool computing is to build systems that can process information and data faster with lower learning costs. This is particularly important in the case of machine learning, as power consumption is often high when training large data sets.

Reserve Pool Calculation Type

• Context Reverb Network
• Echo State Network
• Liquid machine
• Nonlinear transient calculation
• Depth reserve pool calculation

1. Contextual reverberation network

In the context reverberation network, the input layer inputs the signal into the high-dimensional dynamic system, and the information in this high-dimensional dynamic system is read out by a trainable single-layer perceptron. It has two types of dynamical systems: one is a recurrent neural network with random weights fixed, and the other dynamical system is a continuous reaction-diffusion system inspired by the Turing morphogenesis model.

2. Echo State Network

The echo state network has a sparsely connected hidden layer. The connectivity of the hidden layer is usually less than 10%. Attempts to drive a larger random weight-fixed recurrent neural network with an input signal, thereby inducing a nonlinear response signal in each neuron in the reservoir, which is then connected to the desired using a trainable linear combination of all response signals output signal.

3. Liquid state machine

The liquid state machine (LSM) uses a pulse neural network. LSM consists of a large number of nodes or neurons. Each neuron receives time-varying input from other neurons and external sources. Due to the repetitive nature of connections, time-varying inputs become spatiotemporal patterns of activation in network nodes. These spatiotemporal patterns of activation are then read out by linear discriminant units.

4. Nonlinear Transient Calculation

When time-varying input signals leave the internal dynamics of a mechanism, these deviations cause transient or temporary changes, and these changes are reflected in the output of the device.

5. Depth reserve pool calculation

With the emergence of the deep reserve pool calculation model, the reserve pool calculation framework began to expand towards deep learning. in order to process temporal data in a hierarchical manner, and also allows to study the role of hierarchical combinations in RNNs.

The above is the detailed content of A review of sample pool calculations in machine learning. For more information, please follow other related articles on the PHP Chinese website!