Django-rest-framework: Exceptions while resolving field variables in HTML templates

Created on 14 Dec 2015 · 16Comments · Source: encode/django-rest-framework

Whilst rendering a serializer in a HTML template using rest_framework.renderers.TemplateHTMLRenderer and {% render_form serializer %} a couple of exceptions occur whilst resolving variables. (I'm using DRF 3.3.2, Django 1.9 and Django’s Template engine).

This is an example serializer.

class ExampleSerializer(serializers.Serializer):
    quantity = serializers.IntegerField(required=True)

However if the keys causing the exceptions are provided then the issue does not occur.

class ExampleSerializer(serializers.Serializer):
    quantity = serializers.IntegerField(required=True, style={'hide_label': False, 'placeholder': '0'})

This is the traceback in question.

Exception while resolving variable 'hide_label' in template 'rest_framework/vertical/input.html'.
Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 879, in _resolve_lookup
    current = current[bit]
KeyError: 'hide_label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 887, in _resolve_lookup
    current = getattr(current, bit)
AttributeError: 'dict' object has no attribute 'hide_label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 894, in _resolve_lookup
    current = current[int(bit)]
ValueError: invalid literal for int() with base 10: 'hide_label'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 901, in _resolve_lookup
    (bit, current))  # missing attribute
django.template.base.VariableDoesNotExist: Failed lookup for key [hide_label] in "{'input_type': 'number', 'renderer': <rest_framework.renderers.HTMLFormRenderer object at 0x1068889e8>, 'base_template': 'input.html', 'template_pack': 'rest_framework/vertical/'}"
Exception while resolving variable 'placeholder' in template 'rest_framework/vertical/input.html'.
Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 879, in _resolve_lookup
    current = current[bit]
KeyError: 'placeholder'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 887, in _resolve_lookup
    current = getattr(current, bit)
AttributeError: 'dict' object has no attribute 'placeholder'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 894, in _resolve_lookup
    current = current[int(bit)]
ValueError: invalid literal for int() with base 10: 'placeholder'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/James/.virtualenvs/example/lib/python3.4/site-packages/django/template/base.py", line 901, in _resolve_lookup
    (bit, current))  # missing attribute
django.template.base.VariableDoesNotExist: Failed lookup for key [placeholder] in "{'input_type': 'number', 'renderer': <rest_framework.renderers.HTMLFormRenderer object at 0x1068889e8>, 'base_template': 'input.html', 'template_pack': 'rest_framework/vertical/'}"

Source

jamesbeith

Most helpful comment

@RyanBalfanz, I think you can get away with something like the following, just telling django.template logger to be quieter and don't send any messages up. It's a tad shorter than specifying the level for each sub-logger that sends into the main django logger: db, request, security, template.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO')
        },
        'django.template': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}

I'm working on a non-DRF Django site so this is probably not a DRF bug. It's only happening in the django.contrib.admin app. The key named in the tracebacks is different depending on the admin page. Context variables are set in lots of places in the admin app, so I'm not sure where exactly the problem is or how to fix it.

lindsay-stevens on 2 Feb 2016

👍5

All 16 comments

Ah, to add... I had my Django log level set to DEBUG which is why these were in my console when previously with INFO they weren't.

jamesbeith on 17 Dec 2015

I notice I get similar tracebacks with django-debug-toolbar when viewing their SQL panel with DEBUG log level for example. You may want to disregard this issue?

jamesbeith on 17 Dec 2015

Missing context variables are logged as DEBUG messages.

https://docs.djangoproject.com/en/1.9/topics/logging/#django-template

jamesbeith on 17 Dec 2015

+1 am now also suddenly suffering this.

rosscdh on 13 Jan 2016

Seems to only exhibit when viewing via the HTML renderer

rosscdh on 13 Jan 2016

upgraded to djangorestframework==3.3.2 fixed it.

rosscdh on 13 Jan 2016

@rosscdh thanks for the feedback !

xordoquy on 13 Jan 2016

I can confirm that this is still happening when using JSONRenderer().
I'm using it in a regular Django view and passing it to a template using {{ my_json|safe }}. I get the same error.

dvf on 22 Jan 2016

I am also getting this, but when viewing django.contrib.admin site pages, so not sure why DRF would be involved. I took a while to even find this issue, but somewhat glad to see I'm not the only one.

Django==1.9.1
django-debug-toolbar==1.4
django-jsonfield==0.9.15
django-localflavor==1.2
django-rest-swagger==0.3.5
djangorestframework==3.3.2
djangorestframework-jwt==1.7.2

The first Exception triggers a runaway Exception chain, look a lot like @jamesbeith's but with a different key.

Exception while resolving variable 'is_popup' in template 'admin/login.html'.
Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 879, in _resolve_lookup
    current = current[bit]
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/context.py", line 77, in __getitem__
    raise KeyError(key)
KeyError: 'is_popup'

(backend) ➜ [email protected] backend git:(master) ✗ forego run python apiserver/manage.py runserver
Hello!
/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/utils/six.py:808: RemovedInDjango110Warning: SubfieldBase has been deprecated. Use Field.from_db_value instead.
  return meta(name, bases, d)

/Users/ryan/.venvs/backend/lib/python3.5/site-packages/jsonfield/fields.py:130: RemovedInDjango110Warning: SubfieldBase has been deprecated. Use Field.from_db_value instead.
  class TypedJSONField(JSONField):

/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/utils/six.py:808: RemovedInDjango110Warning: SubfieldBase has been deprecated. Use Field.from_db_value instead.
  return meta(name, bases, d)

/Users/ryan/.venvs/backend/lib/python3.5/site-packages/jsonfield/fields.py:130: RemovedInDjango110Warning: SubfieldBase has been deprecated. Use Field.from_db_value instead.
  class TypedJSONField(JSONField):

Performing system checks...

System check identified no issues (0 silenced).
(0.003)
            SELECT c.relname, c.relkind
            FROM pg_catalog.pg_class c
            LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
            WHERE c.relkind IN ('r', 'v')
                AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
                AND pg_catalog.pg_table_is_visible(c.oid); args=None
(0.001) SELECT "django_migrations"."app", "django_migrations"."name" FROM "django_migrations"; args=()
January 29, 2016 - 07:13:22
Django version 1.9.1, using settings 'apiserver.settings.local'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
(0.001) SELECT "django_site"."id", "django_site"."domain", "django_site"."name" FROM "django_site" WHERE "django_site"."id" = 1; args=(1,)
Exception while resolving variable 'is_popup' in template 'admin/login.html'.
Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 879, in _resolve_lookup
    current = current[bit]
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/context.py", line 77, in __getitem__
    raise KeyError(key)
KeyError: 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 885, in _resolve_lookup
    if isinstance(current, BaseContext) and getattr(type(current), bit):
AttributeError: type object 'RequestContext' has no attribute 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 894, in _resolve_lookup
    current = current[int(bit)]
ValueError: invalid literal for int() with base 10: 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 901, in _resolve_lookup
    (bit, current))  # missing attribute
django.template.base.VariableDoesNotExist: Failed lookup for key [is_popup] in '[{\'False\': False, \'None\': None, \'True\': True}, {\'request\': <WSGIRequest: GET \'/admin/login/?next=/admin/\'>, \'csrf_token\': <SimpleLazyObject: <function csrf.<locals>._get_val at 0x10bbb7b70>>, \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'ERROR\': 40, \'WARNING\': 30, \'SUCCESS\': 25, \'INFO\': 20}, \'user\': <SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x10bba03c8>>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x10bba0518>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x10bbe2cf8>}, {}, {\'next\': \'/admin/\', \'LANGUAGE_CODE\': \'en-us\', \'site_title\': <django.utils.functional.lazy.<locals>.__proxy__ object at 0x10a92eb00>, \'available_apps\': [], \'title\': \'Log in\', \'site\': <Site: 127.0.0.1:8000>, \'site_url\': \'/\', \'has_permission\': False, \'site_header\': \'ShiftMedical Administration::{"settings": "local"}\', \'LANGUAGE_BIDI\': False, \'site_name\': \'127.0.0.1:8000\', \'app_path\': \'/admin/login/?next=/admin/\', \'form\': <AdminAuthenticationForm bound=False, valid=Unknown, fields=(username;password)>}]'
Exception while resolving variable 'is_popup' in template 'admin/login.html'.
Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 879, in _resolve_lookup
    current = current[bit]
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/context.py", line 77, in __getitem__
    raise KeyError(key)
KeyError: 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 885, in _resolve_lookup
    if isinstance(current, BaseContext) and getattr(type(current), bit):
AttributeError: type object 'RequestContext' has no attribute 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 894, in _resolve_lookup
    current = current[int(bit)]
ValueError: invalid literal for int() with base 10: 'is_popup'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/template/base.py", line 901, in _resolve_lookup
    (bit, current))  # missing attribute
django.template.base.VariableDoesNotExist: Failed lookup for key [is_popup] in '[{\'False\': False, \'None\': None, \'True\': True}, {\'request\': <WSGIRequest: GET \'/admin/login/?next=/admin/\'>, \'csrf_token\': <SimpleLazyObject: <function csrf.<locals>._get_val at 0x10bbb7b70>>, \'DEFAULT_MESSAGE_LEVELS\': {\'DEBUG\': 10, \'ERROR\': 40, \'WARNING\': 30, \'SUCCESS\': 25, \'INFO\': 20}, \'user\': <SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x10bba03c8>>, \'messages\': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x10bba0518>, \'perms\': <django.contrib.auth.context_processors.PermWrapper object at 0x10bbe2cf8>}, {}, {\'next\': \'/admin/\', \'LANGUAGE_CODE\': \'en-us\', \'site_title\': <django.utils.functional.lazy.<locals>.__proxy__ object at 0x10a92eb00>, \'available_apps\': [], \'title\': \'Log in\', \'site\': <Site: 127.0.0.1:8000>, \'site_url\': \'/\', \'has_permission\': False, \'site_header\': \'ShiftMedical Administration::{"settings": "local"}\', \'LANGUAGE_BIDI\': False, \'site_name\': \'127.0.0.1:8000\', \'app_path\': \'/admin/login/?next=/admin/\', \'form\': <AdminAuthenticationForm bound=False, valid=Unknown, fields=(username;password)>}]'
[29/Jan/2016 07:13:29] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 1728
[29/Jan/2016 07:13:30] "GET /static/admin/css/login.css HTTP/1.1" 200 1203
[29/Jan/2016 07:13:30] "GET /static/admin/css/base.css HTTP/1.1" 200 15897
[29/Jan/2016 07:13:30] "GET /static/admin/css/fonts.css HTTP/1.1" 200 423
[29/Jan/2016 07:13:30] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 200 81348
[29/Jan/2016 07:13:30] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 200 80304
/Users/ryan/.venvs/backend/lib/python3.5/site-packages/django/shortcuts.py:45: RemovedInDjango110Warning: The context_instance argument of render_to_string is deprecated.
  using=using)

[29/Jan/2016 07:13:30] "GET /favicon.ico HTTP/1.1" 200 20457

RyanBalfanz on 29 Jan 2016

@RyanBalfanz DRF isn't mentioned in your traceback, so it's a totally unrelated issue.
@vanflymen unless we have a simple test case we can run - preferably in the DRF test suite - there's no way anyone but you can do anything.

xordoquy on 29 Jan 2016

👎1

@xordoquy, that's right. This was the only other place I could find something somewhat close to what I was seeing, so thought it was worth adding to the conversation. What I meant by "not sure why DRF would be involved" was just what you're saying.

As @jamesbeith already pointed out. These debug messages may very well have existed before, but we just didn't notice until setting DJANGO_LOG_LEVEL=DEBUG.

@jamesbeith Are you using a logging config similar to the second example in the Django docs (i.e. see all of Django’s debug logging which is very verbose)?

import os

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        },
    },
}

RyanBalfanz on 29 Jan 2016

😕1 👍1

@RyanBalfanz Yes, these are new in Django 1.9 when the level is set to DEBUG.

Missing context variables are logged as DEBUG messages.
https://docs.djangoproject.com/en/1.9/topics/logging/#django-template

https://github.com/django/django/blob/cfda1fa3f8d95f0f4a369da9021dbd770e5fa44a/django/template/base.py#L911

A logging settings example.

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': 'DEBUG',
            'propagate': False,
        },
    },
}

jamesbeith on 29 Jan 2016

👍1

@jamesbeith, Cool that's what I expected.

Not to hijack this issue but what I'm really wondering now is, Did those log messages exist before and I just didn't know, or are they actually new.

I think it's probably the case that this is not caused by DRF, given what I'm seeing from the admin site. Given that I'm seeing it from the Django admin, I'm going to guess that the issue already existed but was not visible to me. But would also be surprised to learn that the admin shipped like that…

EDIT: I'd _like_ to be able to set DJANGO_LOG_LEVEL=DEBUG in a local or dev environment, but it's too noisy now with those template errors.

RyanBalfanz on 2 Feb 2016

👍2

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO')
        },
        'django.template': {
            'handlers': ['console'],
            'level': 'INFO',
            'propagate': False,
        },
    },
}

lindsay-stevens on 2 Feb 2016

👍5

@lindsay-stevens-kirby Thanks for sharing. Also, TEMPLATE_DEBUG might be help.

Default: False

Deprecated since version 1.8:
Set the 'debug' option in the OPTIONS of a DjangoTemplates backend instead.
A boolean that turns on/off template debug mode. If this is True, the fancy error page will display a detailed report for any exception raised during template rendering. This report contains the relevant snippet of the template, with the appropriate line highlighted.

Note that Django only displays fancy error pages if DEBUG is True, so you’ll want to set that to take advantage of this setting.

See also DEBUG.

RyanBalfanz on 3 Feb 2016

If you add lots of boilerplate junk to your serializers.py ala:
revoked = serializers.BooleanField( style={ 'hide_label': False, 'placeholder': None } )

You can resolve most of these "errors". It may be we just need to add default values for these to the various default serializers; I haven't tried that approach yet.