Home  >  Article  >  Backend Development  >  How to Replace Empty Strings (Whitespace) with NaN in a Pandas DataFrame?

How to Replace Empty Strings (Whitespace) with NaN in a Pandas DataFrame?

Linda Hamilton
Linda HamiltonOriginal
2024-10-31 04:18:30603browse

How to Replace Empty Strings (Whitespace) with NaN in a Pandas DataFrame?

Replacing Blank Values (White Space) with NaN in Pandas

How can you efficiently replace blank values (whitespace) with NaN in a Pandas dataframe?

Initial Approach:

The following code is capable of replacing blank values with None, but it is inefficient and not the most Pythonic solution:

<code class="python">for i in df.columns:
    df[i][df[i].apply(lambda i: True if re.search('^\s*$', str(i)) else False)] = None</code>

Optimal Solution:

Pandas offers a more concise and efficient solution through the df.replace() method:

<code class="python">df = pd.DataFrame([
    [-0.532681, 'foo', 0],
    [1.490752, 'bar', 1],
    [-1.387326, 'foo', 2],
    [0.814772, 'baz', ' '],     
    [-0.222552, '   ', 4],
    [-1.176781,  'qux', '  '],         
], columns='A B C'.split(), index=pd.date_range('2000-01-01','2000-01-06'))

# replace field that's entirely space (or empty) with NaN
print(df.replace(r'^\s*$', np.nan, regex=True))</code>

This code replaces blank values (regular expressions: ^s*$) with NaN, producing the desired output:

                   A    B   C
2000-01-01 -0.532681  foo   0
2000-01-02  1.490752  bar   1
2000-01-03 -1.387326  foo   2
2000-01-04  0.814772  baz NaN
2000-01-05 -0.222552  NaN   4
2000-01-06 -1.176781  qux NaN

Note:

If you need to handle valid data that may contain white spaces, you can modify the regular expression to r'^s $', which only matches fields consisting entirely of white space.

The above is the detailed content of How to Replace Empty Strings (Whitespace) with NaN in a Pandas DataFrame?. For more information, please follow other related articles on the PHP Chinese website!

Statement:
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn