delete many documents from mongodb

Question

I work on multitenant web application. It's necessary clear some container of users, which can be rather large, having many documents in collection. I need to be able to delete many documents something like: ``` return self::remove(array('LISTID' => $listId), array('safe' => true)); ``` In some cases there can be many documents that meets this criteria, something like 100s of thousands or even millions. I worry that this operation can take much time and throttle server. If there are many documents, is it worth queue such operation to delete them offline something like pseudocode: ``` while (there are documents) { delete(1000 documents); sleep(); } ``` I wonder how to delete data by smaller portios in mongodb in this case. I also notice that for some reason deleting rather many rows takes place rather fast in mongodb, we have prototype with storing data in mongodb, deleting similar number of rows takes much longer in mysql, but in mysql each row in table has references to other table w…

Accepted Answer

This is something that you will have to do in your application. In PHP, you could f.e. do something like:

```
$found = false;
$ids = $collection->find(array('LISTID' => $listId), array('_id' => 1))->limit(1000);
do {
    $found = 0;
    $idsToDelete = array(); // we'll collect all the ids here, so that we can delete them in a batch
    foreach( $ids as $res )
    {
        $found++;
        $idsToDelete[] = $res['_id'];
    }
    $collection->remove(array('_id' => array( '$in' => $idsToDelete )));
    sleep(15);
} while ( $found );
```

You need to make really sure that you have an index on LISTID,otherwise the `find(array('LISTID' => $listId)` could make things really slow.

delete many documents from mongodb

Solution

Problem