Mapping string categories to numbers using pandas and numpy

Question

I have an array of data, each row represents a sample of data (5 samples) and each column represents a feature in the data (6 features for each sample) I'm trying to quantify the number of states each column contains, then map them to a set of numbers. This should only be done if the column is not currently numeric. This is easier to explain through example: example input (Input is of type numpy.ndarray): ``` In = array([['x', 's', 3, 'k', 's', 'u'], ['x', 's', 2, 'n', 'n', 'g'], ['b', 's', 0, 'n', 'n', 'm'], ['k', 'y', 1, 'w', 'v', 'l'], ['x', 's', 2, 'o', 'c', 'l']], dtype=object) ``` For first column ``` curr_column = 0 colset = set() for row in In: curr_element = row[curr_column] if curr_element not in colset: colset.add(curr_element) #now colset = {'x', 'b', 'k'} so 3 possible states collist = list(colset) #make it indexible coldict = {} for i in range(len(collist)): coldict[collist[i]] = i ``` This produces a dictionary, so that I can now recreate the original data as such: (ass…

Accepted Answer

You can use Categorical codes. See Categorical section of the docs.

```
In [11]: df
Out[11]:
   0  1  2  3  4  5
0  x  s  3  k  s  u
1  x  s  2  n  n  g
2  b  s  0  n  n  m

In [12]: for col in df.columns:
     ...:     df[col] = pd.Categorical(df[col], categories=df[col].unique()).codes

In [13]: df
Out[13]:
   0  1  2  3  4  5
0  0  0  0  0  0  0
1  0  0  1  1  1  1
2  1  0  2  1  1  2
3  2  1  3  2  2  3
4  0  0  1  3  3  3
```

I suspect there's a small change which would allow doing this without passing the categories explicitly (Note: pandas does guarantee that `.unique()` is in seen-order).

Note: To "differentiate between columns that are numeric and ones that aren't" you can use `select_dtypes` before iterating:

```
for col in df.select_dtypes(exclude=['int']).columns:
    ...
```

Mapping string categories to numbers using pandas and numpy

Solution

Problem