Django Custom field's attributes making database queries

custom-fields, database, django

Solution

Problem is with your `prepare_value()` method which does explicit queries. `.get()` does not get cached and always hits the db while iterating on `.filter()` queryset will evaluate that. This might be causing you multiple queries.

This is not seen in default fields because they do not do any queries in `prepare_value()`.

To resolve this, you can try to cache the `value` and `result`. If `value` hasn't changed, return cached result. Something like:

class CustomRelatedField(forms.Field):
    def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs):
       self.cached_result = None
       self.cached_value = None
    ...
    def prepare_value(self, value):
       #check we have cached result
       if value == self.cached_value:
           return self.cached_result

       if self.multiple:
           result  = [obj.name for obj in self.model.filter(pk__in=value)]
       else:
           result = self.model.object.get(pk=value)

       #cache the result and value
       self.cached_result = result
       self.cached_value = value    
       return result

Not sure how good/bad this work around though!

Problem

I am facing a very weird problem in one of my django projects. In my project I have a custom field class that handles foreign keys, one to one and many 2 many model fields. The class is some thing like the following. ``` from django import forms class CustomRelatedField(forms.Field): def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs): self.model = model self.limit = limit self.multiple = multiple self.create_objects = create_objects super(CustomRelatedField, self).__init__(*args, **kwargs) def clean(self, value): """ Calls self.get_objects to get the actual model object instance(s) from the given unicode value. """ # Do some value processing here return self.get_objects(value) def get_objects(self, values): """ Returns the model object instances for the given unicode values. """ results = [] for value in values: try: obj = self.model.object.get_or_create(name=value)[0] results.append(obj) except Exception, err: # Log the error here. return results def prepare_value(self, value): """ Returns the value to be sent to the UI. The value passed to this method is generally the object id or a list of object id's (in case it is a many to many object). So we need to make a database query to get the object and then return the name attribute. """ if self.multiple: result = [obj.name for obj in self.model.filter(pk__in=value)] else: result = self.model.object.get(pk=value) return result ``` Recently while I was playing with the django-toolbar, I found out one of the pages that has a form with the above mentioned fields was ridiculously making multiple queries for the same objects again and again. While debugging, I found out the `prepare_value` method was being called again and again. After some more debugging, I realized the culprit was the template. I have a generic template that I use for forms, It looks something like the following: ``` {% for field in form %} {% if field.is_hidden %} <!-- Do something here --> {% endif %} {% if field.field.required %} <!-- Do something here --> {% endif %} <label>{{ field.label }}</label> <div class="form-field">{{ field }}</div> {% if field.field.widget.attrs.help_text %} <!-- Do something here --> {% elif field.errors %} <!-- Do something here --> {% endif %} {% endfor %} ``` In the above code, each `if` statement calls the field class which calls the `prepare_value` method which then makes the database queries. Each of the following listed is making a database query, I am totally lost to why this is happening and have no clue about any solutions. Any help, suggestions would be really appreciated. Thanks. - field.is_hidden - field.field.required - field.label - field.label_tag - field - field.field.widget.attrs.help_text - field.errors Also, why does this happen with my custom field class only, other fields (FKs, O2Os, M2M's) in the application and the application admin, just make one query, even though they are using a similar template.

Original source