Django-rest-framework: Django ValidationError message transformed incorrectly when we use DRF's ValidationError

[Gurubol]

Checklist

• [*] I have verified that that issue exists against the master branch of Django REST framework.
• [*] I have searched for similar issues in both open and closed tickets and cannot find a duplicate.
• [*] This is not a usage question. (Those should be directed to the discussion group instead.)
• [*] This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• [*] I have reduced the issue to the simplest possible case.
• [ ] I have included a failing test as a pull request. (If you are unable to do so we can still accept the issue.)

Steps to reproduce

from django.core.exceptions import ValidationError as DjangoValidationError
from rest_framework.serializers import ValidationError

error = DjangoValidationError('Sample error message', code='error_code')
drf_error = ValidationError(error.error_list)

drf_error.get_full_details()

Expected behavior

The error message should be rendered correctly as follows:

[{u'code': 'error_code', u'message': u"Sample error message"}]

Actual behavior

The message is assumed as a tuple/list and then force_text() renders it incorrectly.

[{u'code': 'error_code', u'message': u"[u'Sample error message']"}]

[Gurubol]

The version I am using is 3.5.3 (latest)

[xordoquy]

Does the documentation mention this should be working ?

[Gurubol]

@xordoquy , I could not locate any documentation but I can give you code snippet which says it works:

https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/exceptions.py#L105 - This says it returns message and code, given the detail (which can nothing but be django.core.exceptions.ValidationError instance)

I could use method as_serializer_error(error), but that poses another problem. If I used field level errors, it ignores them and throws them as non_field_errors. Following is the code snippet:

error2 = DjangoValidationError({'my_field': ValidationError('Another sample error message', code='error_code2')})
drf_error2 = ValidationError(detail=serializers.as_serializer_error(error2))
drf_error2.get_full_details()

{u'non_field_errors': [{u'code': u'invalid',
   u'message': u"[u'Another sample error message']"}]}

It completely ingores my field and assumes that it is a non-field error.

Please let me know if I am missing something.

[Gurubol]

@xordoquy , @tomchristie ,
Hope I have given enough information to consider this as a bug. Appreciate some inputs on this.

[xordoquy]

https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/exceptions.py#L105 - This says it returns message and code, given the detail (which can nothing but be django.core.exceptions.ValidationError instance)

And so it does. In this case, the issue is that the Django ValidationError doesn't translates well.

str(error.error_list[0])
"['Sample error message']"

This isn't a bug but might be improved.

Before going further I would be interested in the bigger picture which is the reason why you're not using regular dict/list structure of strings ?

[Gurubol]

@xordoquy , Thanks for your immediate response. We are trying to use standard format for exception and generally, code is used as a standard way of communication with the other systems which use our APIs. So, instead of just dict/list of strings, we want to use dict/list of ValidationError objects. Hence the need.

[xordoquy]

I figured that but this is still a bit abstract for me as I don't do that.

[tomchristie]

Afraid I'm going to close this off as not intended usage.

[Gurubol]

@tomchristie , If it is not, what is the valid usage. You could see my test case. Please suggest me the right approach.

[Gurubol]

@tomchristie , I just want to re-iterate that this is only after the introduction of ValidationError's code attribute. A standard system nowadays understands code better than the message to take action and/or render appropriate error message. So, if you are saying usage of code is not an intended usage, I would have to disagree. This is definitely a bug.

[rpkilby]

@Gurubol - DRF ValidationErrors accept lists of strings. However, Django's ValidationError.error_list returns a list of ValidationErrors. This isn't compatible, and you need to convert this into a list of strings manually. eg, drf_error = serializers.ValidationError([e.message for e in error.error_list]).

@tomchristie - FWIW, this is an issue that we've also ran across with django-filter. Raising a filterset's underlying form errors is not currently useful, since the Dj validation errors are not caught by the DRF exception handler. Wrapping the Dj error in the DRF equivalent isn't sufficient either, for the same reasons that @Gurubol also discovered. This PR attempts to solve this by converting a nested Dj validation error into a primitive structure of plain dicts & lists, and then reraising as DRF validation error.

It would be fairly straightforward to adapt the above PR to DRF, but:

• Should Django ValidationErrors be promoted to DRF ValidationErrors? eg, support
  python dj_error = forms.ValidationError('Sample error message', code='error_code') drf_error = serializers.ValidationError(dj_error)
• If not, should there be documentation that provides guidance on how to do so?

[Gurubol]

@rpkilby , Thanks for your response and insights. Just to add another point, are we not losing the additional information like ValidationError's code when we do drf_error = serializers.ValidationError([e.message for e in error.error_list])? The main reason why this issue occurs is when you want to keep the code also intact apart from the message for Djanogo's ValidationError. Since, DRF's ValidationError also supports code, should we not seamlessly pass on the information to the newly created object?

[rpkilby]

@Gurubol - the example was to just demonstrate that error_list does not return a data structure that is compatible with DRF's ValidationError. You can provide more complex error handling if necessary.

[Gurubol]

@tomchristie , @rpkilby , I still believe that this is a bug. Please let me know if this is not with suggestions/documentation to handle my use case (show the error messages with the code for ValidationError).

[blueyed]

Please consider re-opening it.
I am seeing the same issue.

My use case is the following in a custom API exception handler:

from rest_framework.serializers import as_serializer_error

if isinstance(exc, DjangoValidationError):
    drf_exception = rest_framework.exceptions.ValidationError(as_serializer_error(exc))

The problem is that get_error_detail removes the information about the fields: https://github.com/encode/django-rest-framework/blob/1c5710bf6caab53012d7a4f0b1b1d84c8a209fd7/rest_framework/serializers.py#L322-L328

[carltongibson]

I’ll have a look at it.

[carltongibson]

First step to progressing this would be an improvement to get_error_detail — or a separate function — that handles the conversion from a Django to a DRF ValidationError.

@rpkilby Has made a start on this here https://github.com/carltongibson/django-filter/pull/702/ — but I'd be happy to see DRF itself host the utility function to handle the conversion. (It seems a common enough need.)

[blueyed]

The following seems to work from a custom exception handler (but also using a custom handler for the DRF exception):

    elif isinstance(exc, DjangoValidationError):
        code = getattr(exc, 'code', None) or 'invalid'
        try:
            message_dict = exc.message_dict
        except AttributeError:
            message_dict = {api_settings.NON_FIELD_ERRORS_KEY: exc.messages}
        logger.info('Django ValidationError (%s): %s (%s)',
                    format(exc), message_dict, code, exc_info=True)
        drf_exception = exceptions.ValidationError(message_dict, code)    elif isinstance(exc, DjangoValidationError):

[blueyed]

Update:

    elif isinstance(exc, DjangoValidationError):
        default_code = getattr(exc, 'code', None) or 'invalid'
        try:
            error_dict = exc.error_dict
        except AttributeError:
            error_dict = {api_settings.NON_FIELD_ERRORS_KEY: exc.error_list}
        logger.info('Django ValidationError (%s): %s',
                    format(exc), error_dict, exc_info=True)

        error_dict = OrderedDict([
            (key, [exceptions.ErrorDetail(e.message % (e.params or ()),
                                          e.code if e.code else default_code)
                   for e in error_list])
            for key, error_list in error_dict.items()])
        drf_validation_error = exceptions.ValidationError(error_dict,
                                                          default_code)
        return api_exception_handler(drf_validation_error, context)

[rpkilby]

I'm removing myself from the ticket, as I don't currently see anything concrete to work on.

In regards to my comment here, I misunderstood a few things. The issue in django-filter is unrelated, as the objective is to transform a form's ErrorDict, not a forms.ValidationError. The issue was complicated by the fact that the ErrorDict was being wrapped inside a ValidationError.

[nwaxiomatic]

Having the same issue when dealing with "unique_together" validation on models

[blueyed]

@nwaxiomatic
Have you tried my snippet in a custom exception handler?

@rpkilby @carltongibson
What do you think about doing this in the default exception handler?

[nwaxiomatic]

@blueyed About to try it, will let you know. Was commenting to find it when I got to my laptop :)

[nwaxiomatic]

Still getting validation response as such:

"abrelationship_set": [
            {
                "non_field_errors": [
                    "The fields a, b must make a unique set."
                ]
            },
]

These errors are being seen as a drf ValidationError within this

def as_serializer_error(exc): 
    assert isinstance(exc, (ValidationError, DjangoValidationError)) 
    if isinstance(exc, ValidationError): 
        detail = exc.detail
    elif isinstance(exc, DjangoValidationError):
        default_code = getattr(exc, 'code', None) or 'invalid'
        try:
            error_dict = exc.error_dict
        except AttributeError:
            error_dict = {api_settings.NON_FIELD_ERRORS_KEY: exc.error_list}
        logger.info('Django ValidationError (%s): %s',
                    format(exc), error_dict, exc_info=True)

        error_dict = OrderedDict([
            (key, [exceptions.ErrorDetail(e.message % (e.params or ()),
                                          e.code if e.code else default_code)
                   for e in error_list])
            for key, error_list in error_dict.items()])
        drf_validation_error = exceptions.ValidationError(error_dict,
                                                          default_code)
        return api_exception_handler(drf_validation_error, context)

    if isinstance(detail, Mapping):
        # If errors may be a dict we use the standard {key: list of values}.
        # Here we ensure that all the values are *lists* of errors.
        return {
            key: value if isinstance(value, (list, Mapping)) else [value]
            for key, value in detail.items()
        }
    elif isinstance(detail, list):
        # Errors raised as a list are non-field errors.
        return {
            api_settings.NON_FIELD_ERRORS_KEY: detail
        }
    # Errors raised as a string are non-field errors.
    return {
        api_settings.NON_FIELD_ERRORS_KEY: [detail]
    }

[blueyed]

"The fields a, b must make a unique set."

Where is it coming from? Not from Django itself, is it?
Usually with multiple fields involved "non_field_errors" is expected.

[nwaxiomatic]

Its a "unique_together" constraint on the model "abrelationship". The issue I am having is within the

"abrelationship_set": [
            {
                "non_field_errors": [
                    "The fields a, b must make a unique set."
                ]
            },
{
                "non_field_errors": [
                    "The fields a, b must make a unique set."
                ]
            },
]

because with the nested serializer it's not telling me which of the abrelationships in abrelationship_set is getting the key error

[blueyed]

@nwaxiomatic
It is not clear to me what your issue really is, but it seems it is unrelated, i.e. it is not about a DjangoValidationError being incorrectly transformed, is it? (since you use non_field_errors yourself already).
Where does "abrelationship_set" come from? Do you set it yourself somewhere?

[nwaxiomatic]

Hey, so finally figured out the deeper issue and this was not part of the issue. I do get back an ordered array with the error in the proper place in the array, with all of the non-errors showing up as empty, like so

"abrelationship_set": [
            {},
            {
                "non_field_errors": [
                    "There already exists a relationship with the same a and b."
                ]
            },
            {},
]

Sorry for the confusion!

[VishvajitP]

We need an update here.

[Allan-Nava]

is possible to get only the error_description without the array .

{
    "error": [
        "invalid_grant"
    ],
    "error_description": [
        "Invalid user credentials"
    ]
}

this is an example but i need to get -->{ "error_description" : "invalid user credentials"}

[willyguevara]

in my case,

i'm getting

{
    "message": [
        "Some message here"
    ],
    "remaining_minutes": [
        "9.89355055"
    ]
}

but i need

{
    "message": "Some message here",
    "remaining_minutes": 9.89355055
}

[xordoquy]

The discussion group is the best place to take this discussion and other usage questions. Thanks!