Remove rows from a pandas dataframe at random without shuffling dataset

dataframe, pandas, python, python-3.x, random

Solution

The following should work for you. Here I sample `remove_n` random row_ids from `df`'s index. After that `df.drop` removes those rows from the data frame and returns the new subset of the old data frame.

import pandas as pd
import numpy as np
np.random.seed(10)

remove_n = 1
df = pd.DataFrame({"a":[1,2,3,4], "b":[5,6,7,8]})
drop_indices = np.random.choice(df.index, remove_n, replace=False)
df_subset = df.drop(drop_indices)

DataFrame `df`:

    a   b
0   1   5
1   2   6
2   3   7
3   4   8

DataFrame `df_subset`:

    a   b
0   1   5
1   2   6
3   4   8

Problem

I've got a dataset which needs to omit a few rows whilst preserving the order of the rows. My idea was to use a mask with a random number between `0` and the length of my dataset but I'm not sure how to setup a mask without shuffling the rows around i.e. a method similar to sampling a dataset. Example: Dataset has 5 rows and 2 columns and I would like to remove a row at random. ``` Col1 Col2 A 1 B 2 C 5 D 4 E 0 ``` transforms to: ``` Col1 Col2 A 1 B 2 D 4 E 0 ``` with the third row (`Col1='C'`) omitted by a random choice. How should I go about this?

Original source