Finding non-numeric rows in dataframe in pandas?

dataframe, pandas, python

Solution

You could use `np.isreal` to check the type of each element (applymap applies a function to each element in the DataFrame):

In [11]: df.applymap(np.isreal)
Out[11]:
          a     b
item
a      True  True
b      True  True
c      True  True
d     False  True
e      True  True

If all in the row are True then they are all numeric:

In [12]: df.applymap(np.isreal).all(1)
Out[12]:
item
a        True
b        True
c        True
d       False
e        True
dtype: bool

So to get the subDataFrame of rouges, (Note: the negation, ~, of the above finds the ones which have at least one rogue non-numeric):

In [13]: df[~df.applymap(np.isreal).all(1)]
Out[13]:
        a    b
item
d     bad  0.4

You could also find the location of the first offender you could use argmin:

In [14]: np.argmin(df.applymap(np.isreal).all(1))
Out[14]: 'd'

As @CTZhu points out, it may be slightly faster to check whether it's an instance of either int or float (there is some additional overhead with np.isreal):

df.applymap(lambda x: isinstance(x, (int, float)))

Problem

I have a large dataframe in pandas that apart from the column used as index is supposed to have only numeric values: ``` df = pd.DataFrame({'a': [1, 2, 3, 'bad', 5], 'b': [0.1, 0.2, 0.3, 0.4, 0.5], 'item': ['a', 'b', 'c', 'd', 'e']}) df = df.set_index('item') ``` How can I find the row of the dataframe `df` that has a non-numeric value in it? In this example it's the fourth row in the dataframe, which has the string `'bad'` in the `a` column. How can this row be found programmatically?

Original source