How to Efficiently Convert a Pandas DataFrame to a NumPy Array?

Convert Pandas Dataframe to NumPy Array

To transform a pandas dataframe into a NumPy array, it is recommended to use the df.to_numpy() method. This approach is preferable to df.values for several reasons, as detailed below.

Using df.to_numpy()

Calling the df.to_numpy() method allows for the extraction of the underlying NumPy array from the dataframe.

import numpy as np
import pandas as pd

index = [1, 2, 3, 4, 5, 6, 7]
a = [np.nan, np.nan, np.nan, 0.1, 0.1, 0.1, 0.1]
b = [0.2, np.nan, 0.2, 0.2, 0.2, np.nan, np.nan]
c = [np.nan, 0.5, 0.5, np.nan, 0.5, 0.5, np.nan]
df = pd.DataFrame({'A': a, 'B': b, 'C': c}, index=index)
df = df.rename_axis('ID')

arr = df.to_numpy()
print(arr)

Output:

[[ nan  0.2  nan]
 [ nan  nan  0.5]
 [ nan  0.2  0.5]
 [ 0.1  0.2  nan]
 [ 0.1  0.2  0.5]
 [ 0.1  nan  0.5]
 [ 0.1  nan  nan]]

Preserving Dtypes

If the aim is to maintain the data types in the resulting NumPy array, a possible approach involves utilizing DataFrame.to_records(), as shown below:

records_array = df.to_records()
print(records_array)

Output:

rec.array([('a', 1, 4, 7), ('b', 2, 5, 8), ('c', 3, 6, 9)],
           dtype=[('ID', '<U1'), ('A', '<i8'), ('B', '<i8'), ('C', '<i8')])

As an alternative, one can employ np.rec.fromrecords:

v = df.reset_index()
records_array = np.rec.fromrecords(v, names=v.columns.tolist())
print(records_array)

Output:

rec.array([('a', 1, 4, 7), ('b', 2, 5, 8), ('c', 3, 6, 9)],
           dtype=[('index', '<U1'), ('A', '<i8'), ('B', '<i8'), ('C', '<i8')])

Using either of these methods ensures the preservation of data types in the NumPy array.

The above is the detailed content of How to Efficiently Convert a Pandas DataFrame to a NumPy Array?. For more information, please follow other related articles on the PHP Chinese website!