How Can I Expand Nested Lists in Pandas DataFrames into Separate Rows?

Unraveling Nested Lists in Pandas DataFrames: Row Expansion

When working with data in Pandas dataframes, you may encounter columns containing lists, potentially spanning multiple values. To facilitate analysis and manipulation, it becomes necessary to transform these lists into separate rows. This process, known as "long forming" or "row expansion," allows each list element to occupy its own row.

In order to achieve this, Pandas offers a dedicated method called .explode(), introduced in version 0.25. This method seamlessly transforms the specified list-containing column into a series of rows, with each element becoming an independent row.

Implementation:

To employ the .explode() method, simply specify the column name you wish to expand. By default, it will create new rows for each element within the column, while preserving the values in all other columns.

For example, consider a dataframe containing a 'samples' column with lists of values:

import pandas as pd
import numpy as np

df = pd.DataFrame(
    {'trial_num': [1, 2, 3, 1, 2, 3],
     'subject': [1, 1, 1, 2, 2, 2],
     'samples': [list(np.random.randn(3).round(2)) for i in range(6)]
    }
)

Applying the .explode() method:

df.explode('samples')

Results in the following dataframe:

   subject  trial_num  sample
0        1          1    0.57
1        1          1   -0.83
2        1          1    1.44
3        1          2   -0.01
4        1          2    1.13
5        1          2    0.36
6        2          1   -0.08
7        2          1   -4.22
8        2          1   -2.05
9        2          2    0.72
10       2          2    0.79
11       2          2    0.53

As you can observe, each list element now has its own row. It is worth noting that, although the method efficiently unrolls the lists, it does so for a single column at a time.

Additional Considerations:

• The .explode() method handles mixed columns containing both lists and scalar values, ensuring that empty lists and NaN values are preserved appropriately.
• If a dataframe has multiple list-containing columns, you can use nested .explode() calls to unroll them one by one.
• If you desire specific column ordering, you can manually readjust the column order after calling .explode().
• Resetting the index using .reset_index(drop=True) is recommended to obtain a regular integer index.

The above is the detailed content of How Can I Expand Nested Lists in Pandas DataFrames into Separate Rows?. For more information, please follow other related articles on the PHP Chinese website!