Detailed explanation of the F1 score indicator for machine learning evaluation

The accuracy metric measures the number of times the model correctly predicted across the entire data set. However, this metric is only reliable if the data set is class balanced. That is, there are the same number of samples for each category in the dataset. However, real-world datasets are often severely imbalanced, rendering accuracy metrics no longer feasible. To solve this problem, F1 score was introduced as a more comprehensive and complete machine learning evaluation metric. The F1 score combines the precision and recall of the model and can better evaluate the accuracy of the model. Precision refers to how many of the samples predicted as positive by the model are true positives, while recall refers to how many true positives the model can correctly predict. The calculation formula of F1 score is: 2 * (precision rate * recall rate) / (precision rate recall rate). By comprehensively considering precision and recall, the F1 score can more accurately evaluate the performance of the model, especially in

F1 score concept

F1 Scores are closely related to confusion matrices and are used to evaluate metrics such as accuracy, precision, and recall of a classifier. By combining precision and recall, the F1 score provides an assessment of the overall performance of the model.

Precision measures how many of the “positive” predictions the model made were correct.

Recall measures how many positive samples present in the data set are correctly recognized by the model.

Precision and recall provide a trade-off relationship, where improving one metric comes at the expense of the other. Higher accuracy means a stricter classifier that will doubt the actual positive samples in the dataset, thus lowering the recall rate. On the other hand, higher recall requires a relaxed classifier that allows any sample similar to the positive class to pass through, which will misclassify some edge-case negative samples as "positive class" and thus reduce accuracy. Ideally, we would like to maximize the precision and recall metrics to obtain a perfect classifier.

The F1 score combines precision and recall using their harmonic mean, maximizing the F1 score means maximizing precision and recall simultaneously.

How to calculate F1 score?

To understand the calculation of F1 score, you first need to understand the confusion matrix. Above we mentioned that the F1 score is defined in terms of precision and recall. The formula is as follows:

Precision

The F1 score is calculated as the harmonic mean of the precision and recall scores as shown below. It ranges from 0-100%, with a higher F1 score indicating better classifier quality.

To calculate the F1 score for a multi-class dataset, a one-to-one technique is used to calculate the individual scores for each class in the dataset. Take the harmonic mean of class precision and recall values. The net F1 score is then calculated using different averaging techniques.

Macro-average F1 score

Micro-average F1 score is a meaningful indicator for multi-class data distribution. It uses "net" TP, FP and FN values ​​to calculate the indicator.

Net TP refers to the sum of the class TP scores of the dataset, which is calculated by decomposing the confusion matrix into a one-vs-all matrix corresponding to each class.

Sample Weighted F1 Score

The Fβ score is a generic version of the F1 score. It calculates the harmonic mean, just like the F1 score, but prioritizes precision or recall. "β" represents the weight coefficient, which is a hyperparameter set by the user and is always greater than 0.

The above is the detailed content of Detailed explanation of the F1 score indicator for machine learning evaluation. For more information, please follow other related articles on the PHP Chinese website!