Pydantic: Validators not called in order when there are default field values

[manugrandio]

Bug

• OS: MacOS 10.14.6
• Python version: 3.6.8
• Pydantic version: 0.32.2

Hi!

About this statement from the docs regarding validators (https://pydantic-docs.helpmanual.io/#validators):

Validation is done in the order fields are defined, eg. here password2 has access to password1 (and name), but password1 does not have access to password2.

I think it doesn't apply when there's a field with a default value. For example, in the following tests UserModel and UserModelOptionalName only differ in that in the latter the name has a default value of None. However validators behave differently:

from unittest import TestCase

from pydantic import BaseModel, validator


@validator('password1')
def name_not_in_password(cls, v, values):
    assert values['name'] not in v
    return v


class UserModel(BaseModel):
    name: str
    username: str
    password1: str
    password2: str

    name_not_in_password = name_not_in_password


class UserModelOptionalName(BaseModel):
    name: str = None
    username: str
    password1: str
    password2: str

    name_not_in_password = name_not_in_password


class Test(TestCase):
    def setUp(self):
        self.data = {
            'name': 'John',
            'username': 'john',
            'password1': '123',
            'password2': '123',
        }

    def test_user_model(self):
        UserModel(**self.data)

    def test_user_model_optional_name(self):
        UserModelOptionalName(**self.data)

Have the following results:

test_user_model (test.Test) ... ok
test_user_model_optional_name (test.Test) ... ERROR

======================================================================
ERROR: test_user_model_optional_name (test.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/manu/tmp/testpydantic/test.py", line 43, in test_user_model_optional_name
    UserModelOptionalName(**self.data)
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/main.py", line 275, in __init__
    values, fields_set, _ = validate_model(__pydantic_self__, data)
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/main.py", line 757, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls or model.__class__)  # type: ignore
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/fields.py", line 317, in validate
    v, errors = self._validate_singleton(v, values, loc, cls)
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/fields.py", line 450, in _validate_singleton
    return self._apply_validators(v, values, loc, cls, self.validators)
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/fields.py", line 457, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
  File "/Users/manu/tmp/testpydantic/.venv/lib/python3.6/site-packages/pydantic/class_validators.py", line 171, in <lambda>
    return lambda cls, v, values, field, config: validator(cls, v, values=values)
  File "/Users/manu/tmp/testpydantic/test.py", line 8, in name_not_in_password
    assert values['name'] not in v
KeyError: 'name'

----------------------------------------------------------------------
Ran 2 tests in 0.004s

FAILED (errors=1)

It looks like when there's a field with a default value, the validation order is altered and name is not in values yet.

[dmontagu]

@manugrandio This was fixed in https://github.com/samuelcolvin/pydantic/pull/715. This used to be addressed in the docs, but the online docs now reflect V1 rather than v0.32.2, which is why the relevant section is now missing.

Sorry for any confusion!

(Note: if you drop the annotation entirely the field will still come at the end; it seems like this would be hard to fix given how python works.)

[manugrandio]

@dmontagu Ok, thanks for your response!

[samuelcolvin]

There a note at the top of the docs with a link to the old docs for v0.32.

[dmontagu]

I was just wrong in my comment above -- the note is still in the docs (and they are still the v0.32.2 docs where I linked). For some reason I couldn't find it earlier when I went to look but I did just now.

Warning

Be aware that using annotation only fields will alter the order of your fields in metadata and errors: annotation only fields will always come first, but still in the order they were defined.

[samuelcolvin]

yes, I've changed this quite a lot in the new docs.