pandas GroupBy columns with NaN (missing) values

group-by, nan, pandas, python

Solution

pandas >= 1.1

From pandas 1.1 you have better control over this behavior, NA values are now allowed in the grouper using `dropna=False`:

pd.__version__
# '1.1.0.dev0+2004.g8d10bfb6f'

# Example from the docs
df

   a    b  c
0  1  2.0  3
1  1  NaN  4
2  2  1.0  3
3  1  2.0  2

# without NA (the default)
df.groupby('b').sum()

     a  c
b        
1.0  2  3
2.0  2  5

# with NA
df.groupby('b', dropna=False).sum()

     a  c
b        
1.0  2  3
2.0  2  5
NaN  1  4

Problem

I have a DataFrame with many missing values in columns which I wish to groupby: ``` import pandas as pd import numpy as np df = pd.DataFrame({'a': ['1', '2', '3'], 'b': ['4', np.NaN, '6']}) In [4]: df.groupby('b').groups Out[4]: {'4': [0], '6': [2]} ``` see that Pandas has dropped the rows with NaN target values. I want to include these rows! Any suggestions?

Original source