Python Pandas: How to filter a dataframe with more than one expression stored in different variables?
expression, filter, pandas, python, variables
Solution
As a newcomer to numpy I struggled a bit (no pun intended) about this too. I believe you want something like this:
>>> df[(df['ID'] == 'p') | (df['ID'] == 'ul')]
The expression must evaluate to a boolean (and the terms must be connected through bitwise operations), which then is used to mask or filter the corresponding elements.
See also:
- http://pandas.pydata.org/pandas-docs/dev/indexing.html#boolean-indexing
- http://docs.scipy.org/doc/numpy/user/basics.indexing.html#boolean-or-mask-index-arrays
- https://stackoverflow.com/a/13572798/89391
- http://pandas.pydata.org/pandas-docs/dev/generated/pandas.Series.isin.html
Problem
I am building a multy purpose User Interface, and I am adding Pandas to it. For this, I need to form expressions by components (stored in variables) which are defined by users choices. All seems to work fine, but I got into a dead end. I want the user to be able to pick several expressions, and then concatenate them to form the new dataframe. If I only use one expression, everything will work: ``` from pandas import read_csv df = read_csv("SomeCsv.csv") b= df[r'ID'] a=(b==r'p') Value=df[a] #Works,returning the rows in df whichs column 'ID' equals r'p' ``` But if I want to include more expressions: ``` from pandas import read_csv df = read_csv("SomeCsv.csv") b= df[r'ID'] c=(b==r'p') d=(b==r'ul') a=c or d #Breaks at this line Value=df[a] #Doesnt work. I would expect the rows in df whichs column 'ID' equals r'p' or 'ID' equals r'ul' ``` And throws the following error: ``` ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ``` Before asking, I tried all the .any and .all combinations of the expressions I could think of, and all of them failed. How to filter this dataframe by columns matching more than one expression stored in variables?