Pymongo w=1 with continue_on_error
mongodb, pymongo
Solution
Even with `continue_on_error=True`, PyMongo will raise a DuplicateKeyError if MongoDB tells it that you tried to insert a document with a duplicate `_id`. However, with `continue_on_error=True`, the server has attempted to insert all the documents in your list, instead of aborting the operation on the first error. The `error_document` attribute of the exception tells you the last duplicate `_id` in your list of documents.
Unfortunately you cannot determine how many documents succeeded and failed in total when you do a bulk insert. MongoDB 2.6 and PyMongo 2.7 will address this in the next release when we implement bulk write operations.
Problem
I have a collection of tweets. I want to insert a list of tweets into this collection. The new list may have some duplicate tweets as well and I want to ensure that duplicate tweets do not get written but all remaining does. To achieve this, I'm using following code. ``` mongoPayload = <list of tweets> committedTweetIDs = db.tweets.insert(mongoPayload, w=1, continue_on_error=True) print "%d documents committed" % len(committedTweetIDs) ``` The above code snippet should work. However, the behavior I'm getting is that second line generated DuplicateKeyError. I don't know what this is happening since, I mentioned continue_on_error. What I want in the end is for Mongo to commit all the non-duplicate documents and return to me (as acknowledgement) tweetIDs of all the documents written to the journal.