Remove rows where value in one column equals value in another
pandas, python
Solution
`Series.ne` (`!=`)
df[df['Column2'] != df['Column4']]
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
Or, using `operator.ne`:
df[operator.ne(df['Column2'], df['Column4'])]
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
Compare the two; get a mask, then filter.
With `loc`, we can also supply a callback (suggested by @W-B!).
df.loc[lambda x : x['Column2'] != x['Column4']]
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
`query`
df.query('Column2 != Column4')
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
`np.vectorize`
import operator
f = pd.np.vectorize(lambda x, y: x != y)
df[f(df['Column2'], df['Column4'])]
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
...Just for fun.
List Comprehension
df[[x != y for x, y in zip(df['Column2'], df['Column4'])]]
Column1 Column2 Column3 Column4
0 Pat 123 John 456
1 Pat 123 John 345
3 Larry 678 James 983
Faster than you think!
Problem
I'm struggling to figure out how to remove rows from a pandas dataframe in which two specified columns have the same value across a row. For example, in the below examples I would like to remove the rows which have duplicate values in the columns 2 and 4. For example: ``` Column1 Column2 Column3 Column4 Pat 123 John 456 Pat 123 John 345 Jimmy 678 Mary 678 Larry 678 James 983 ``` Would turn into: ``` Column1 Column2 Column3 Column4 Pat 123 John 456 Pat 123 John 345 Larry 678 James 983 ``` Any help is appreciated, thank you!