MongoDB to store different (schema) log files
logging, mongodb
Solution
Obviously I'm biased (I work on MongoDB) but I think it works very well for logs.
Reasons:
- It's fast for inserts and updates... you can do thousands per second
- As well as normal queries, you can run analytics and generate reports using JavaScript. You could have a cron job running nightly which does nice MapReduce things to your logs.
- You can use capped collections, which are collection that act like queues, to keep only the latest N KBs/MBs/GBs of logs
I'm not sure what you mean "ensure the integrity of a log file"... do you mean you are worried about not knowing what fields the document you're pulling out has? If so, I think you'll find it's no harder dealing with null fields in a relational database and much more flexible.
See also: the MongoDB blog post on logging.
Problem
Do you think that using a MongoDB Json Database to store log files from application is a good idea and why ? The only advantage for me is the schema abstraction, but i think it's also a weakness we cannot ensure the integrity of a log file.