Convert Django Model object to dict with all of the fields intact

dictionary, django, django-models, django-orm, python

Solution

There are many ways to convert an instance to a dictionary, with varying degrees of corner case handling and closeness to the desired result.

1. `instance.__dict__`

instance.__dict__

which returns

{'_foreign_key_cache': <OtherModel: OtherModel object>,
 '_state': <django.db.models.base.ModelState at 0x7ff0993f6908>,
 'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is by far the simplest, but is missing `many_to_many`, `foreign_key` is misnamed, and it has two unwanted extra things in it.

2. `model_to_dict`

from django.forms.models import model_to_dict
model_to_dict(instance)

which returns

{'foreign_key': 2,
 'id': 1,
 'many_to_many': [<OtherModel: OtherModel object>],
 'normal_value': 1}

This is the only one with `many_to_many`, but is missing the uneditable fields.

3. `model_to_dict(..., fields=...)`

from django.forms.models import model_to_dict
model_to_dict(instance, fields=[field.name for field in instance._meta.fields])

which returns

{'foreign_key': 2, 'id': 1, 'normal_value': 1}

This is strictly worse than the standard `model_to_dict` invocation.

4. `query_set.values()`

SomeModel.objects.filter(id=instance.id).values()[0]

which returns

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key_id': 2,
 'id': 1,
 'normal_value': 1,
 'readonly_value': 2}

This is the same output as `instance.__dict__` but without the extra fields. `foreign_key_id` is still wrong and `many_to_many` is still missing.

5. Custom Function

The code for django's `model_to_dict` had most of the answer. It explicitly removed non-editable fields, so removing that check and getting the ids of foreign keys for many to many fields results in the following code which behaves as desired:

from itertools import chain

def to_dict(instance):
    opts = instance._meta
    data = {}
    for f in chain(opts.concrete_fields, opts.private_fields):
        data[f.name] = f.value_from_object(instance)
    for f in opts.many_to_many:
        data[f.name] = [i.id for i in f.value_from_object(instance)]
    return data

While this is the most complicated option, calling `to_dict(instance)` gives us exactly the desired result:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

6. Use Serializers

Django Rest Framework's ModelSerializer allows you to build a serializer automatically from a model.

from rest_framework import serializers
class SomeModelSerializer(serializers.ModelSerializer):
    class Meta:
        model = SomeModel
        fields = "__all__"

SomeModelSerializer(instance).data

returns

{'auto_now_add': '2018-12-20T21:34:29.494827Z',
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

This is almost as good as the custom function, but auto_now_add is a string instead of a datetime object.

Bonus Round: better model printing

If you want a django model that has a better python command-line display, have your models child-class the following:

from django.db import models
from itertools import chain

class PrintableModel(models.Model):
    def __repr__(self):
        return str(self.to_dict())

    def to_dict(instance):
        opts = instance._meta
        data = {}
        for f in chain(opts.concrete_fields, opts.private_fields):
            data[f.name] = f.value_from_object(instance)
        for f in opts.many_to_many:
            data[f.name] = [i.id for i in f.value_from_object(instance)]
        return data

    class Meta:
        abstract = True

So, for example, if we define our models as such:

class OtherModel(PrintableModel): pass

class SomeModel(PrintableModel):
    normal_value = models.IntegerField()
    readonly_value = models.IntegerField(editable=False)
    auto_now_add = models.DateTimeField(auto_now_add=True)
    foreign_key = models.ForeignKey(OtherModel, related_name="ref1")
    many_to_many = models.ManyToManyField(OtherModel, related_name="ref2")

Calling `SomeModel.objects.first()` now gives output like this:

{'auto_now_add': datetime.datetime(2018, 12, 20, 21, 34, 29, 494827, tzinfo=<UTC>),
 'foreign_key': 2,
 'id': 1,
 'many_to_many': [2],
 'normal_value': 1,
 'readonly_value': 2}

Problem

How does one convert a django Model object to a dict with all of its fields? All ideally includes foreign keys and fields with editable=False. Let me elaborate. Let's say I have a django model like the following: ``` from django.db import models class OtherModel(models.Model): pass class SomeModel(models.Model): normal_value = models.IntegerField() readonly_value = models.IntegerField(editable=False) auto_now_add = models.DateTimeField(auto_now_add=True) foreign_key = models.ForeignKey(OtherModel, related_name="ref1") many_to_many = models.ManyToManyField(OtherModel, related_name="ref2") ``` In the terminal, I have done the following: ``` other_model = OtherModel() other_model.save() instance = SomeModel() instance.normal_value = 1 instance.readonly_value = 2 instance.foreign_key = other_model instance.save() instance.many_to_many.add(other_model) instance.save() ``` I want to convert this to the following dictionary: ``` {'auto_now_add': datetime.datetime(2015, 3, 16, 21, 34, 14, 926738, tzinfo=<UTC>), 'foreign_key': 1, 'id': 1, 'many_to_many': [1], 'normal_value': 1, 'readonly_value': 2} ``` Questions with unsatisfactory answers: Django: Converting an entire set of a Model's objects into a single dictionary How can I turn Django Model objects into a dictionary and still have their foreign keys?

Original source

Related problems