Why Does `UndefinedMetricWarning` Appear Only Once When Calculating F1 Score?

UndefinedMetricWarning: F-score Undefined for Labels with No Predicted Samples

In scikit-learn, the f1_score metric calculates the F1 score, which measures the accuracy of a classification model. However, the calculation requires the presence of predicted samples for each label in the target variable (y_test). If certain labels are missing from the predicted samples (y_pred), the F1 score for those labels becomes undefined, resulting in an error message:

UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.

Why the Error Occurs Only Once

Although the error message suggests an undefined metric, it is actually a warning that appears only once by default. This is because Python treats warnings differently from errors. By default, most environments display specific warnings only once, even if the underlying condition persists.

Resolving the UndefinedMetricWarning

To resolve the warning, it is crucial to ensure that all labels in y_test are also present in y_pred. This can be determined by comparing the sets of labels:

set(y_test) - set(y_pred)

If the result is an empty set, all labels have been predicted, and there will be no undefined F1 scores.

Avoiding Repeated Warnings

If it is important to see the warning every time it occurs, the warnings.filterwarnings() function can be used to modify the warning handling behavior:

import warnings
warnings.filterwarnings('always')

Ignoring Labels with No Predicted Samples

Alternatively, specific labels can be excluded from the F1 score calculation by specifying the labels of interest:

metrics.f1_score(y_test, y_pred, average='weighted', labels=np.unique(y_pred))

This ensures that labels missing from the predicted samples are not considered in the score calculation, eliminating the warning.

The above is the detailed content of Why Does `UndefinedMetricWarning` Appear Only Once When Calculating F1 Score?. For more information, please follow other related articles on the PHP Chinese website!