Introduction to vector norms in machine learning: L1, L2, and L∞ norms

Vector norm is a measure of vector size and is widely used to evaluate model error. Plays an important role in machine learning and deep learning.

A machine learning project can be viewed as an n-dimensional vector, where each dimension represents an attribute of the data. Therefore, we can calculate the distance between them using standard vector-based similarity measures, such as Manhattan distance, Euclidean distance, etc. Simply put, the norm is a function that helps us quantify the size of a vector.

Properties of vector norm

Vector norm satisfies the following 4 properties:

• Non-negativity : is always non-negative.
• Deterministic: It is zero only when the vector is zero
• Triangle inequality: The norm of the sum of two vectors does not exceed the sum of their norms.
• Homogeneity: Multiply a vector by a scalar Multiply the norm of the vector by the absolute value of the scalar.

Common vector norms in machine learning

L1 norm

The symbol of the L1 norm is ||v||1 calculates the Manhattan distance from the origin to the vector space, and the L1 norm calculates the sum of absolute vector values. In machine learning, we usually use L1 norm when the sparsity of vectors is important.

Formula: ||v||1= |b1| |b2| |b3|

L2 norm

The symbol of the L2 norm is ||v||2. This norm is also called the Euclidean norm. The L2 norm is calculated as the square root of the sum of the squared values ​​of the vector. Since it is differentiable Function, L2 norm is most commonly used for optimization in machine learning.

Formula: ||v||2= sqrt [ (b1)2 (b2)2 (b3)2]

Vector maximum Norm

The symbol of the maximum norm is ||v||inf, or the infinity symbol can be used to represent L∞. The maximum norm is calculated as the maximum value of the returned vector.

Formula: ||v||inf= max( |b1| , |b2| , |b3| )

Many applications , such as information retrieval, personalization, document classification, image processing, etc., all rely on the calculation of similarity or dissimilarity between items. Two items are considered similar if the distance between them is small and vice versa.

The above is the detailed content of Introduction to vector norms in machine learning: L1, L2, and L∞ norms. For more information, please follow other related articles on the PHP Chinese website!