Data preprocessing in Python (code)

The content this article brings to you is about data preprocessing (code) in Python. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.

1. Import standard library

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

2. Import data set

dataset = pd.read_csv('data (1).csv')  # read_csv：读取csv文件
#创建一个包含所有自变量的矩阵，及因变量的向量
#iloc表示选取数据集的某行某列；逗号之前的表示行，之后的表示列；冒号表示选取全部，没有冒号，则表示选取第几列；values表示选取数据集里的数据。
X = dataset.iloc[:, :-1].values # 选取数据，不选取最后一列。
y = dataset.iloc[:, 3].values # 选取数据,选取每行的第3列数据

3. Missing data

from sklearn.preprocessing import Imputer #进行数据挖掘及数据分析的标准库，Imputer缺失数据的处理
#Imputer中的参数：missing_values 缺失数据，定义怎样辨认确实数据，默认值：nan ;strategy 策略，补缺值方式 ： mean-平均值 ， median-中值 ， most_frequent-出现次数最多的数 ； axis =0取列 =1取行 
imputer = Imputer(missing_values = 'NaN', strategy = 'mean', axis = 0)
imputer = imputer.fit(X[:, 1:3])#拟合fit
X[:, 1:3] = imputer.transform(X[:, 1:3])

4. Classified data

from sklearn.preprocessing import LabelEncoder,OneHotEncoder
labelencoder_X=LabelEncoder()
X[:,0]=labelencoder_X.fit_transform(X[:,0])
onehotencoder=OneHotEncoder(categorical_features=[0])
X=onehotencoder.fit_transform(X).toarray()
#因为Purchased是因变量，Python里面的函数可以将其识别为分类数据，所以只需要LabelEncoder转换为分类数字
labelencoder_y=LabelEncoder()
y=labelencoder_y.fit_transform(y)

5. Divide the data set into a training set and a test set

from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=0)
#X_train（训练集的字变量）,X_test（测试集的字变量）,y_train（训练集的因变量）,y_test（训练集的因变量）
#训练集所占的比重0.2~0.25，某些情况也可分配1/3的数据给训练集；train_size训练集所占的比重
#random_state决定随机数生成的方式，随机的将数据分配给训练集和测试集；random_state相同时会得到相同的训练集和测试集

6. Feature scaling

#特征缩放（两种方式：一：Standardisation(标准化）；二：Normalisation（正常化））
from sklearn.preprocessing import StandardScaler
sc_X=StandardScaler()
X_train=sc_X.fit_transform(X_train)#拟合,对X_train进行缩放
X_test=sc_X.transform(X_test)#sc_X已经被拟合好了，所以对X_test进行缩放时，直接转换X_test

7. Data preprocessing template

(1) Import standard library
(2) Import data set
(3) Few missing and classified items Encounter
(4) Split the data set into a training set and a test set
(5) Feature scaling is not needed in most cases, but feature scaling is required in some cases

The above is the detailed content of Data preprocessing in Python (code). For more information, please follow other related articles on the PHP Chinese website!