Google App Engine NDB: How to store document structure?

Question

From App Engine NDB documentation: The NDB API provides persistent storage in a schemaless object datastore. It supports automatic caching, sophisticated queries, and atomic transactions. NDB is well-suited to storing structured data records. I want to create a structure like the following using NDB, where each instance looks like : ``` { city: 'SFO' date: '2013-01-27' data: { 'keyword1': count1, 'keyword2': count2, 'keyword3': count3, 'keyword4': count4, 'keyword5': count5, .... } } ``` How can I design such a schema-less entity in Google App Engine(GAE) using NDB? I am new to GAE and not sure how to achieve this Thank you

Accepted Answer

If you don't need to query for the attributes in data you can use one of the properties as mentioned by @voscausa:

JsonProperty

```
class MyModel(ndb.Model):
  city = ndb.StringProperty()
  date = ndb.DateProperty()
  data = ndb.JsonProperty()

my_model = MyModel(city="somewhere", 
                   date=datetime.date.today(),
                   data={'keyword1': 3,
                         'keyword2': 5,
                         'keyword3': 1,})
```

StructuredProperty:

```
class Data(ndb.Model):
  keyword = ndb.StringProperty()
  count = ndb.IntegerProperty()

class MyModel(ndb.Model):
  city = ndb.StringProperty()
  date = ndb.DateProperty()
  data = ndb.StructuredProperty(Data, repeated=True)

my_model = MyModel(city="somewhere", 
                   date=datetime.date.today(),
                   data=[Data(keyword="keyword1", count=3),
                         Data(keyword="keyword2", count=5),
                         Data(keyword="keyword3", count=1)])
my_model.put()
```

The problem here is filtering for structured properties. The properties of Keyword are viewed as parallel arrays. Doing a query such as:

```
q = MyModel.query(MyModel.data.keyword=='keyword1',
                  MyModel.data.count > 4)
```

would incorrectly include `my_model`.

https://developers.google.com/appengine/docs/python/ndb/queries#filtering_structured_properties

Using an expando model would work and allow you to query for keywords:

```
class MyModel(ndb.Expando):
  city = ndb.StringProperty()
  date = ndb.DateProperty()

m = MyModel(city="Somewhere", date=datetime.date.today())
m.keyword1 = 3
m.keyword2 = 5
m.keyword3 = 1
m.put()

q = MyModel.query(ndb.GenericProperty('keyword1') > 2) 
```

https://developers.google.com/appengine/docs/python/ndb/entities#expando

Google App Engine NDB: How to store document structure?

Solution

Problem