delete duplicate values in a row of pandas dataframe

numpy, pandas, python

Solution

use a combination of `astype(bool)` and `duplicated`

mask = df_freq.apply(pd.Series.duplicated, 1) & df_freq.astype(bool)

df_freq.mask(mask, 0)

     A  B    C
0  Z11  0  X11
1  Y11        
2  Z11  0

Problem

I have a pandas data frame: ``` >>df_freq = pd.DataFrame([["Z11", "Z11", "X11"], ["Y11","",""], ["Z11","Z11",""]], columns=list('ABC')) >>df_freq A B C 0 Z11 Z11 X11 1 Y11 2 Z11 Z11 ``` I want to make sure each row has unique values only. Therefore it should become like this: Removed values can be replaced with zero or empty ``` A B C 0 Z11 0 X11 1 Y11 2 Z11 0 ``` My data frame is big with hundreds of columns and thousands of rows. The goal is to count the unique values in that data frame. I do that by using converting data frame to matrix and applying ``` >>np.unique(mat.astype(str), return_counts=True) ``` But in certain row(s) the same value occurs and I want to remove that before applying np.unique() method. I want to keep unique values in each row.

Original source