Application of common loss functions in twin neural networks

Twin neural network is a neural network with a dual-branch structure, which is often used for similarity measurement, classification and retrieval tasks. The two branches of such a network have the same structure and parameters. After the input passes through two branches respectively, the similarity is calculated through the similarity measurement layer (such as Euclidean distance, Manhattan distance, etc.). During training, a contrastive loss function or a triplet loss function is usually used.

The contrast loss function is a binary classification loss function for twin neural networks. It aims to maximize the similarity of similar samples to be close to 1 and to reduce the similarity of different types of samples. adjusted to close to 0. Its mathematical expression is as follows:

L_{con}(y,d)=y\cdot d^2 (1-y)\cdot\max(m-d,0)^2

This loss function is used to measure the similarity between two samples and is optimized according to the category of the sample. Among them, y represents whether the sample belongs to the same category, d represents the similarity of the two samples, and m represents a preset boundary value. When y=1, the goal of the loss function is to make d as small as possible, even if two samples of the same category are more similar. At this time, the value of the loss function can be expressed by the square of d, that is, the value of the loss function is d^2. When y=0, the goal of the loss function is to make d greater than m, even if the samples of two different categories are as dissimilar as possible. At this time, when d is less than m, the value of the loss function is d^2, which indicates the similarity between samples; when d is greater than m, the value of the loss function is 0, which indicates that the similarity between samples has exceeded the predetermined level. Set the boundary value m, and no longer calculate the loss

The triplet loss function is a loss function used in twin neural networks, aiming to minimize the distance between similar samples , and maximize the distance between samples of different classes. The mathematical expression of this function is as follows:

L_{tri}(a,p,n)=\max(|f(a)-f(p)|^2- |f(a)-f(n)|^2 margin,0)

Among them, a represents the anchor sample, p represents the same type of sample, n represents the different type of sample, f represents The feature extraction layer of the twin neural network, |\cdot| represents the Euclidean distance, and margin represents a preset boundary value. The goal of the loss function is to make the distance between samples of the same type as small as possible, and the distance between samples of different types as large as possible, and larger than the margin. When the distance between samples of the same type is less than the distance of samples of different types minus margin, the value of the loss function is 0; when the distance of samples of the same type is greater than the distance of samples of different types minus margin, the value of the loss function is the difference between the two distances. .

Contrast loss function and triplet loss function are commonly used twin neural network loss functions. Their goal is to make samples of the same category as close as possible in the feature space, and samples of different categories The samples are as far apart as possible in the feature space. In practical applications, an appropriate loss function can be selected based on specific tasks and data sets, and combined with other techniques (such as data enhancement, regularization, etc.) for model optimization.

The above is the detailed content of Application of common loss functions in twin neural networks. For more information, please follow other related articles on the PHP Chinese website!