Django-rest-framework: Additional (non model) fields in ModelSerializer -> TypeError (inconsistent with Django forms)

[medihack]

As discussed here: https://groups.google.com/forum/?fromgroups=#!topic/django-rest-framework/yZdpt85g9is

When using additional fields in a ModelSerializer (see below for an example) that are not available as attributes in the model itself, it will throw an TypeError (full error below) during deserialization as the serializer seems to force to build an object with that attribute. Django forms are much more tolerant in this case. If one deletes that field from the attrs array in clean_additional_info everything works ok.

I often have the case that I need additional stuff in an API post (create) request that is not directly represented in the model (but changes several other attributes in the model). It would be nice if Django Rest Framework behaves here more like Django Forms.

Example code:

# My serializer
class PostSerializer(serializers.ModelSerializer):
    additional_info = serializers.WritableField()  # no corresponding model property

    class Meta:
        model = Post
        fields = ('title', 'content', 'additional_info',)

    def validate_additional_info(self, attrs, source):
        value = attrs[source]

        # do something with value (maybe adjust existing model properties)

       del attrs[source]  # would be nice if this were enough
       return attrs

# The exception
  File "/home/user/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/site-packages/rest_framework/serializers.py", line 787, in restore_object
    instance = self.opts.model(**attrs)
  File "/home/user/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/site-packages/django/db/models/base.py", line 415, in __init__
    raise TypeError("'%s' is an invalid keyword argument for this function" % list(kwargs)[0])
TypeError: 'additional_info' is an invalid keyword argument for this function

[tomchristie]

Marking for DjangoCon sprints attention. Basically just needs someone else to investigate and report on what you think is sensible. Is this valid, is this already achievable by some other route etc... ?

[tomchristie]

Seems reasonable, yup.

[tomchristie]

May be that the behaviour changed since the version you were using when you reported this, but this is already supported as specified in this ticket. Added a regression test, and closing this issue.

[vbabiy]

I think this only solve half the problem, because if you ever call serializer.data you will get a AttributeError. That Post object has no attribute 'additional_info'.

It almost seems you need two serializers, one to handle the input and one to handle the output. Doing so doesn't make it very DRY.

[philipn]

I think this issue may still be here. The problem seems like it isn't the validation; it's the presence of an attribute in a WritableField that isn't present on the model while using ModelSerializer. My workaround was to use restore_object on the main model serializer to pop the attribute:

https://github.com/localwiki/localwiki/blob/6caa98178299bd51f0f296a96350aeff89e05b0c/localwiki/pages/serializers.py#L18

In this case, tags isn't on Page. If the pop() is removed, we'll see a TypeError. I'm pretty new to DRF, so I'm not sure if this could be simplified without introducing additional complexity.

The reason why this seemed a bit confusing to me is because, with ModelForms, you can have attributes that aren't on the model and it doesn't try to pass them to the model's constructor.

[tomchristie]

Thanks for the follow up @philipn - let's reopen this for now then.

[artursoler]

I think the real issue here is that, for consistency with Django Forms, you should not need to add your extra field to the Meta's fields tuple. Currently if you don't add it it's ignored by the serializer.

[Atala]

@ArturSoler : I am quite surprised by your comment, as adding an extra field to the Meta's tuple will throw an attribute error. Or maybe this behaviour has changed ?

Is it possible to add extra fields ? If not, it may be good to edit the documentation on this point.

EDIT : it works, but for trying to do something like 'serializer.data' will throw a-not-so-nice AttributeError. Is this the only problem remaining for adding extra fields?

Alois

[dozymoe]

I have a patch that whitelist model attributes to meta.local_fields https://github.com/dozymoe/django-rest-framework/commit/f8f1964bb15baf789fc252655b0aaee4bb721945
I'm noob so I don't know what the side effects will be.

My problem was GenericRelation not listed as m2m_relations or something.

[tomchristie]

We've made the decision to close of all currently open serializer tickets, pending the 3.0 release.

The 3.0 release will involve a major redesign and improvement of the current serializer implementation and should invalidate the large majority of these outstanding tickets.

We will be reviewing the closed tickets prior to the 3.0 launch and try to ensure that we have fully covered any long-standing issues, and adequately dealt with the outstanding problems.

If your issue does not appear to be addressed by the upcoming 3.0 release please do comment on your ticket with the current status, and we can reopen any valid tickets as required. In particular, if your ticket represents a long-standing or fundamental problem please do make sure to follow the mailing list and review any 3.0 pre-release candidates where possible.

Note that you can review the closed tickets with the following searches:

Serializer tickets: https://github.com/tomchristie/django-rest-framework/issues?q=label%3ASerializers
Writable nested serializer tickets: https://github.com/tomchristie/django-rest-framework/issues?q=label%3A%22Writable+nested+serializers%22

Many thanks for your understanding and here's looking forward to a new and improved version, and a cleaner more actionable issue list.

All the best,

Tom

[gitaarik]

For people that end up here still looking for a way, check out this stack overflow answer:
https://stackoverflow.com/a/37718821/1248175