Pandas: fillna(None) inconsistent with fillna(1) or fillna(np.NaN)

Created on 26 Sep 2012  路  2Comments  路  Source: pandas-dev/pandas

Migrated from StackOverflow comment : http://stackoverflow.com/questions/12598520/set-index-on-multiple-columns-with-one-empty-column

    df = DataFrame([
    dict(a=1, p=0), 
    dict(a=2, m=10), 
    dict(a=3, m=11, p=20), 
    dict(a=4, m=12, p=21)
    ], columns=('a', 'm', 'p', 'x'))
. a m p x
0 1 NaN 0 NaN
1 2 10 NaN NaN
2 3 11 20 NaN
3 4 12 21 NaN

Applying fillna(None) turns one value to 0 !

df.fillna(None)

. a m p x
0 1 NaN 0 NaN
1 2 10 0 NaN
2 3 11 20 NaN
3 4 12 21 NaN

Applying fillna(1) works as expected :

df.fillna(1)

. a m p x
0 1 1 0 1
1 2 10 1 1
2 3 11 20 1
3 4 12 21 1

Applying fillna(np.NaN) works as expected :

df.fillna(np.NaN)

. a m p x
0 1 NaN 0 NaN
1 2 10 NaN NaN
2 3 11 20 NaN
3 4 12 21 NaN

Could be related to #1971

Most helpful comment

Note: you can do this using the where method:

In [10]: df.where(pd.notnull(df), None)
Out[10]: 
   a     m     p     x
0  1  None     0  None
1  2    10  None  None
2  3    11    20  None
3  4    12    21  None

All 2 comments

Note that fillna(None) is equivalent to fillna(), which means the value parameter is unused. Instead, it uses the method parameter which is by default forward fill. Which is why the NaN after the 0 in column p is filled.

Note: you can do this using the where method:

In [10]: df.where(pd.notnull(df), None)
Out[10]: 
   a     m     p     x
0  1  None     0  None
1  2    10  None  None
2  3    11    20  None
3  4    12    21  None
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jreback picture jreback  路  61Comments

simonjayhawkins picture simonjayhawkins  路  53Comments

datapythonista picture datapythonista  路  44Comments

Dr-Irv picture Dr-Irv  路  59Comments

ghost picture ghost  路  84Comments