How can I query mongodb using mongoid/rails without timing out?

Question

I have a rake task that processes a set of records and saves it in another collection: ``` batch = [] Record.where(:type => 'a').each do |r| batch << make_score(r) if batch.size %100 == 0 Score.collection.insert(batch) batch = [] end end ``` I'm processing about 100K records at a time. Unfortunately at 20 minutes, I get a `Query response returned CURSOR_NOT_FOUND` error. The mongodb faq says to use `skip` and `limit` or turn off timeouts, using them the all thing was about ~2-3 times slower. How can I turn off timeouts in conjunction with mongoid?

Accepted Answer

The MongoDB docs say you can pass in a timeout boolean, and it timeout is false, it will never timeout

```
collection.find({"type" => "a"}, {:timeout=>false})
```

In your case:

```
Record.collection.find({:type=>'a'}, :timeout => false).each ...
```

I also recommend you look into map-reduced with Mongo. It seems tailer made to this sort of collection array manipulation: http://www.mongodb.org/display/DOCS/MapReduce

How can I query mongodb using mongoid/rails without timing out?

Solution

Problem