Python database WITHOUT using Django (for Heroku)

Question

To my surprise, I haven't found this question asked elsewhere. Short version, I'm writing an app that I plan to deploy to the cloud (probably using Heroku), which will do various web scraping and data collection. The reason it'll be in the cloud is so that I can have it be set to run on its own every day and pull the data to its database without my computer being on, as well as so the rest of the team can access the data. I used to use AWS's SimpleDB and DynamoDB, but I found SDB's storage limitations to be to small and DDB's poor querying ability to be a problem, so I'm looking for a database system (SQL or NoSQL) that can store arbitrary-length values (and ideally arbitrary data structures) and that can be queried on any field. I've found many database solutions for Heroku, such as ClearDB, but all of the information I've seen has shown how to set up Django to access the database. Since this is intended to be script and not a site, I'd really prefer not to dive into Django if I don'…

Accepted Answer

I'd use MongoDB. Heroku has support for it, so I think it will be really easy to start and scale out: https://addons.heroku.com/mongohq

About Python: MongoDB is a really easy database. The schema is flexible and fits really well with Python dictionaries. That's something really good.

You can use PyMongo

```
from pymongo import Connection
connection = Connection()

# Get your DB
db = connection.my_database

# Get your collection
cars = db.cars

# Create some objects
import datetime
car = {"brand": "Ford",
       "model": "Mustang",
       "date": datetime.datetime.utcnow()}

# Insert it
cars.insert(car)
```

Pretty simple, uh?

Hope it helps.

EDIT:

As Endophage mentioned, another good option for interfacing with Mongo is mongoengine. If you have lots of data to store, you should take a look at that.

Python database WITHOUT using Django (for Heroku)

Solution

Problem