Deep neural networks for classification calibration and regression calibration

Deep neural network is a powerful machine learning model that can automatically learn features and patterns. However, in practical applications, the output of neural networks often needs to be calibrated to improve performance and reliability. Classification calibration and regression calibration are commonly used calibration techniques, and they have different principles and applications. This article will introduce in detail the working principles and application scenarios of these two technologies.

1. Classification calibration

Classification calibration is to improve the reliability and accuracy of the classifier by adjusting the probability distribution of the classifier output vector to achieve. In a classification problem, a neural network outputs a vector representing the predicted probability of each class. However, these probabilities are not always accurate and may be biased too high or too low. The goal of classification calibration is to adjust these probabilities to make them closer to the true probability distribution. This improves the performance of the classifier, making it more reliable in predicting the probabilities of different classes.

Commonly used classification calibration methods include the following two:

1. Temperature scaling

Temperature scaling is a simple and effective classification calibration technique. Its principle is to adjust the confidence of the classifier by scaling the output of the neural network. Specifically, temperature scaling introduces a temperature parameter T to scale the output of the last layer of the neural network, converting the original prediction probability p into the calibrated probability q:

q_i=\frac{p_i^{\frac{1}{T}}}{\sum_{j=1}^K p_j^{\frac{1}{T}}}

where, i represents the th i categories, K represents the total number of categories. When T=1, temperature scaling will not change the original prediction probability. When T>1, temperature scaling will increase the confidence of the classifier, making the prediction probability more concentrated and confident; when T

The advantages of temperature scaling are that it is simple to implement, low cost, and can be calibrated without retraining the model. However, temperature scaling is not guaranteed to effectively improve the performance of the classifier in all cases because it assumes that the errors of all categories are independent and on the same scale, which is not necessarily true in practical applications.

2.Platt calibration (Platt scaling)

Platt calibration is a relatively traditional classification calibration method, and its principle is based on logistic regression Model, fit the output of the neural network to obtain the calibrated probability distribution. Specifically, Platt calibration introduces a binary logistic regression model to fit the output of the neural network to obtain a new probability distribution. The input of the logistic regression model is the output or feature of the neural network, and the output is a probability value between 0 and 1. By fitting a logistic regression model, the corrected probability value for each category can be obtained.

The advantage of Platt calibration is that it can more accurately estimate the prediction probability and is suitable for a variety of different classification problems. However, Platt calibration requires model fitting, so the computational cost is high, and a large amount of labeled data is required to train the logistic regression model.

2. Regression calibration

Regression calibration refers to correcting the output of the regression model to make it more reliable and accurate. In regression problems, the output of a neural network is usually a continuous real value that represents the value of the predicted target variable. However, there may be bias or variance in these predicted values, requiring regression calibration to improve the accuracy and reliability of the predictions.

Commonly used regression calibration methods include the following two:

1. Historical Average Calibration

Historical average calibration is a simple and effective regression calibration technique. Its principle is to use historical data to calculate the mean and variance of the target variable, and then adjust the predicted value of the neural network. Specifically, historical average calibration obtains a calibration factor by calculating the mean and variance of historical data, and then corrects the predicted value of the neural network to make it closer to the true target value. The advantage of historical average calibration is that it is simple and easy to use, does not require additional training data and computational costs, and is suitable for a variety of different regression problems.

2. Linear Regression Calibration

Linear regression calibration is a regression calibration technology based on a linear regression model. The principle is to map the predicted value of the neural network to the real target value by fitting a linear model. Specifically, linear regression calibration uses additional labeled data to train a linear regression model, taking the predicted value of the neural network as input and the true target value as the output to obtain a linear mapping function, thereby performing the prediction on the neural network's predicted value. calibration.

The advantage of linear regression calibration is that it can more accurately estimate the relationship between the predicted value and the target value, and is suitable for various regression problems. However, the fitting of linear regression models requires a large amount of labeled data and computational costs, and may not be effective for regression problems with nonlinear relationships.

In general, classification calibration and regression calibration are common calibration techniques in deep neural networks, which can improve the performance and reliability of the model. Classification calibration mainly adjusts the confidence of the classifier to make the prediction probability more accurate; regression calibration mainly makes the prediction results closer to the true target value by correcting the bias and variance of the predicted value. In practical applications, appropriate calibration methods need to be selected according to specific problems and combined with other techniques to optimize the performance of the model.

The above is the detailed content of Deep neural networks for classification calibration and regression calibration. For more information, please follow other related articles on the PHP Chinese website!