Django-filter: [2.1.0] BaseCSVWidget.value_from_datadict — AttributeError: 'list' object has no attribute 'split'

Created on 12 Jul 2019 · 15Comments · Source: carltongibson/django-filter

I ran into …

Traceback (most recent call last):
  [..]
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'

and found that the code at …
https://github.com/carltongibson/django-filter/blob/1f47e36b614724a8735e0457fa511dcaf5448481/django_filters/widgets.py#L195-L202
… is not robust with regard to super().value_from_datadict(data, files, name) returning a list. I need to debug this more but maybe you already know what's going on here?

Source

moseb

Most helpful comment

I just had this error and its explanation is buried in this thread, so to recap and save a long search for future readers:

ManyToMany fields declared in viewsets' filterset_fields must not contain in

A warning at runtime would be very appreciated.

marcosox on 26 Jun 2020

👍9

All 15 comments

Hi @moseb. Can you paste the FilterSet code that you're using? The last few lines of the trace back are sufficient to determine why you'd hit this exception.

rpkilby on 12 Jul 2019

👍1

Full traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.7/site-packages/django/contrib/staticfiles/handlers.py", line 65, in __call__
    return self.application(environ, start_response)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 141, in __call__
    response = self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 75, in get_response
    response = self._middleware_chain(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django_global_request/middleware.py", line 15, in __call__
    return self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 145, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 143, in _get_response
    response = response.render()
  File "/usr/local/lib/python3.7/site-packages/django/template/response.py", line 106, in render
    self.content = self.rendered_content
  File "/usr/local/lib/python3.7/site-packages/rest_framework/response.py", line 72, in rendered_content
    ret = renderer.render(self.data, accepted_media_type, context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 733, in render
    context = self.get_context(data, accepted_media_type, renderer_context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 710, in get_context
    'filter_form': self.get_filter_form(data, view, request),
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 642, in get_filter_form
    html = backend().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/rest_framework_filters/backends.py", line 52, in to_html
    return super().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 105, in to_html
    return template.render(context, request)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 199, in render
    c = self.get_render(context).flatten()
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 118, in get_render
    actual_form.form_html = helper.render_layout(actual_form, node_context, template_pack=self.template_pack)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/helper.py", line 308, in render_layout
    template_pack=template_pack
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 140, in render
    return self.get_rendered_fields(form, form_style, context, template_pack, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in get_rendered_fields
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in <genexpr>
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/utils.py", line 148, in render_field
    html = template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 302, in render
    match = condition.eval(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 876, in eval
    return self.value.resolve(context, ignore_failures=True)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 671, in resolve
    obj = self.var.resolve(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 796, in resolve
    value = self._resolve_lookup(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 837, in _resolve_lookup
    current = getattr(current, bit)
  File "/usr/local/lib/python3.7/site-packages/django/forms/boundfield.py", line 74, in errors
    return self.form.errors.get(self.name, self.form.error_class())
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 180, in errors
    self.full_clean()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 381, in full_clean
    self._clean_fields()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'
[12/Jul/2019 13:49:06] "GET /XXXXXXXX/?ordering=first_name HTTP/1.1" 500 59

moseb on 12 Jul 2019

It's from browsing a listing where django-rest-framework generates the code.

moseb on 12 Jul 2019

Hi @moseb. Can you paste the FilterSet code that you're using?

To be honest, I haven't found any in that area of the code, yet. It might be all by DRF. Not sure yet.
I'm aware my bug report is not ideal but it's the best I have right now. Sorry!

moseb on 12 Jul 2019

Your API view should have either a filterset_class or filterset_fields, or if you're using an older version of django-filter, filter_class and filter_fields.

rpkilby on 12 Jul 2019

👍1

I found use of filter_class — so I guess you just helped me find that the migration to django-filter 2.0 was not properly done to its end. Cool!
Is there something stopping django-filter from detecting pre-2.x leftovers and warning about them, conceptually? Does it warn but I missed it? Would code warning about these be a welcome addition for a pull request?

moseb on 12 Jul 2019

v2.0 removed a lot of the existing deprecation warnings, so you might try running your test suite against v1.0 and then v1.1 of django-filter.

rpkilby on 12 Jul 2019

👍1

Assuming this will resolve itself when you update. Come back if not.

carltongibson on 13 Jul 2019

👍1

There is reproducible demo of the issue now at https://github.com/moseb/django-filter-issue-1103-demo. Please consider re-opening this issue. Thank you!

moseb on 19 Jul 2019

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

rpkilby on 19 Jul 2019

👍1

Hi!

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

Is there existing tickets on GitHub for these issues?

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

PS: Can we re-open this ticket?

moseb on 22 Jul 2019

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

Possibly. If the value is already a list, it should just go ahead and return that list. That said, I don't think it makes sense to mix the CSV-behavior with the SelectMultiple widget.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

The ModelMultipleChoiceFilter constructs an OR query from Q objects. So, if you have a query string like /api/mymodel?m2m=a&m2m=b, you would end up with a filter call like

MyModel.objects.filter(Q(m2m='a') | Q(m2m='b'))

With the 2.1 release, the lookup_expr is now applied to each Q object, so you'd end up with

MyModel.objects.filter(Q(m2m__in='a') | Q(m2m__in='b'))

The above is invalid and would break, however it makes sense for contains and other lookups that are intended to work with single values.

So, in short, all you need to do here is use the exact lookup for your m2m field.

I think the todo here is:

Make BaseCSVWidget compatible with SelectMultiple by returning the value if it's already a list. Or if it shouldn't be compatible, we should at least provide a useful error on init.
Fix Meta.fields for m2m. Generating an in lookup for ModelMultipleChoiceFilter is nonsensical. We should also test what happens for other lookups like isnull, which expects a boolean.

rpkilby on 22 Jul 2019

Yeah, this still doesn't look like a bug but how it all works is under documented I guess...

carltongibson on 23 Jul 2019

👍1

I just had this error and its explanation is buried in this thread, so to recap and save a long search for future readers:

ManyToMany fields declared in viewsets' filterset_fields must not contain in

A warning at runtime would be very appreciated.

marcosox on 26 Jun 2020

👍9

@marcosox Good idea. Happy to look at a PR adding that!

carltongibson on 27 Jun 2020

Was this page helpful?

0 / 5 - 0 ratings

Related issues

Allow to override retrieving of a queryset in a FilterSet

Alexx-G · 4Comments

Could anyone give an example to use with Django/Postgres native Full Text Search engine?

gotexis · 4Comments

The django_filters.BooleanFilter with method and a boolean filter inherited from a model don't work on a same way.

inakrin · 3Comments

Filter fields are technically allowed for all DRF view actions. This messes with CoreAPI schema.

Sonictherocketman · 3Comments

Filtering ArrayField

sassanh · 4Comments