Ridge regression example in Python

Ridge regression is a commonly used linear regression method. It can achieve better results than ordinary least squares regression when dealing with multicollinearity problems, and can also be used for feature selection.

Python is a powerful programming language, and it is very convenient to use Python for ridge regression analysis. This article will introduce how to use Python to perform ridge regression analysis through an example.

First, we need to import the required libraries, as shown below:

import pandas as pd
import numpy as np
from sklearn.linear_model import Ridge
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

The data used in this example is Boston housing price data. This data set contains 13 different housing prices in the Boston area in the 1970s. Information on features and their prices. We can read the data in through the read_csv function in the pandas library, as shown below:

data = pd.read_csv('Boston.csv')

Next, we need to divide the data set into a training set and a test set. This can be achieved using the train_test_split function in the scikit-learn library, as shown below:

X = data.iloc[:, :-1].values
y = data.iloc[:, -1].values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

Before training the ridge regression model, we need to normalize the data to ensure that the value ranges of different features vary greatly. can compare their effects on the target variable. We can use the StandardScaler function in the scikit-learn library for standardization, the code is as follows:

from sklearn.preprocessing import StandardScaler
sc_X = StandardScaler()
X_train = sc_X.fit_transform(X_train)
X_test = sc_X.transform(X_test)

Then we can define a ridge regression model and put it into our training data set for training, the code is as follows Shown:

ridge = Ridge(alpha=0.1)
ridge.fit(X_train, y_train)

The alpha value is a hyperparameter, and the model needs to be optimized by adjusting parameters. We can choose the optimal hyperparameters by evaluating the predictions on the training and test sets. In this example, we choose to perform cross-validation on the alpha value to select the optimal hyperparameters. The code is as follows:

from sklearn.model_selection import GridSearchCV
ridge_params = {'alpha': [0.001, 0.01, 0.1, 1, 10]}
ridge_grid = GridSearchCV(estimator=Ridge(), param_grid=ridge_params, cv=10, scoring='neg_mean_squared_error')
ridge_grid.fit(X_train, y_train)
print("Best alpha:", ridge_grid.best_params_['alpha'])

Through cross-validation, we choose the optimal alpha value to be 0.1.

Next, we can make predictions on the test set and evaluate the prediction results. We can use the mean_squared_error function in the scikit-learn library to calculate the mean square error, the code is as follows:

y_pred = ridge.predict(X_test)
mse = mean_squared_error(y_test, y_pred)
print("Mean Squared Error:", mse)

Finally, we can use the matplotlib library to draw a scatter plot of predicted values ​​and true values ​​for better Understand the prediction effect of the model. The code is as follows:

import matplotlib.pyplot as plt
plt.scatter(y_test, y_pred)
plt.xlabel("True Values")
plt.ylabel("Predictions")
plt.show()

In short, ridge regression analysis in Python is very convenient. Using the functions of the scikit-learn library can help us easily evaluate and visualize the prediction results.

The above is the detailed content of Ridge regression example in Python. For more information, please follow other related articles on the PHP Chinese website!