Data Wrangling Techniques in Python

Python is a high-level programming language widely used in the field of data science. It is widely used in data collection, cleaning, analysis and visualization. Data wrangling is a core skill in data processing. This article will introduce some common data wrangling techniques in Python to help readers better process and analyze data.

1. Data type conversion

In the process of data regularization, it is often necessary to convert different data types. Common data types include strings, integers, and floating point numbers. and Boolean values ​​etc. Python provides powerful type conversion functions, such as int(), float(), str(), bool(), etc., which can convert one data type to another data type, for example:

# 将字符串转换成整数
age_str = '18'
age_int = int(age_str)

# 将整数转换成字符串
age_int = 18
age_str = str(age_int)

# 将浮点数转换成整数
height_float = 1.75
height_int = int(height_float)

# 将整数转换成布尔值
num = 0
is_zero = bool(num)     # False

2. Data deduplication

When processing a large amount of data, duplicate data may occur, and data deduplication techniques need to be used. Using the set() function in Python can quickly remove duplicate elements from the list, for example:

# 去除列表中的重复元素
lst = [1, 2, 3, 2, 4, 1]
lst_unique = list(set(lst))
print(lst_unique)       # [1, 2, 3, 4]

3. Data filling

In the process of data regularization, sometimes it is necessary to Missing values ​​are filled for better subsequent processing. Use the fillna() function in Python to easily fill data, for example:

# 对缺失值进行填充
import pandas as pd

df = pd.DataFrame({'name': ['Alice', 'Bob', 'Charlie'],
                   'age': [18, None, 21],
                   'gender': ['F', 'M', None]})

df_fill = df.fillna(value={'age': df['age'].mean(),
                           'gender': 'U'})
print(df_fill)

The output results are as follows:

       name   age gender
0     Alice  18.0      F
1       Bob  19.5      M
2  Charlie  21.0      U

4. Data reshaping

In During the data curation process, data may need to be reshaped for better subsequent processing. Using the pivot() function in Python can easily reshape data, for example:

# 数据重塑
import pandas as pd

df = pd.DataFrame({'name': ['Alice', 'Bob', 'Charlie'],
                   'gender': ['F', 'M', 'M'],
                   'subject': ['Math', 'Math', 'English'],
                   'score': [90, 87, 88]})

df_res = df.pivot(index='name', columns='subject', values='score')
print(df_res)

The output results are as follows:

subject  English  Math
name                  
Alice        NaN  90.0
Bob          NaN  87.0
Charlie     88.0   NaN

5. Data merge

In In actual operations, data is usually stored in different tables and needs to be merged. Using the merge() function in Python can facilitate data merging, for example:

# 数据合并
import pandas as pd

df1 = pd.DataFrame({'name': ['Alice', 'Bob', 'Charlie'],
                    'age': [18, 19, 21],
                    'gender': ['F', 'M', 'M']})
df2 = pd.DataFrame({'name': ['Alice', 'Bob'],
                    'score': [90, 87]})

df_merge = pd.merge(df1, df2, on='name')
print(df_merge)

The output result is as follows:

       name  age gender  score
0     Alice   18      F     90
1       Bob   19      M     87

In summary, data shaping skills in Python include data type conversion, Data deduplication, data filling, data reshaping and data merging, etc. These techniques can help readers better process and analyze data and improve the efficiency and accuracy of data processing.

The above is the detailed content of Data Wrangling Techniques in Python. For more information, please follow other related articles on the PHP Chinese website!