Pydantic: validation does not include expected fields when a field has default value

[kognate]

Bug

Please complete:

• OS: Linux (ubuntu 19.04)
• Python version import sys; print(sys.version):

3.7.3 (default, Aug 20 2019, 17:04:43)
[GCC 8.3.0]

• Pydantic version import pydantic; print(pydantic.VERSION):

0.32.2

Please read the docs and search through issues to
confirm your bug hasn't already been reported before.

Where possible please include a self contained code snippet describing your bug:

The code below shows the UserModel (which is used in the docs to describe custom validation and complex relationships. The problem is that if one of the fields has a default value, it cannot be found in the values dictionary of the decorated method. This means relationship validations can only be performed on fields that do not have default values (and it's confusing).

from pydantic import BaseModel, ValidationError, validator
import pytest


class UserModel(BaseModel):
    name: str
    password1: str = "hello" 
    password2: str

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()

    @validator("password2")
    def passwords_match(cls, v, values, **kwargs):
        if "password1" in values and v != values["password1"]:
            raise ValueError("passwords do not match")
        return v

class UserModelNoDefaultValue(BaseModel):
    name: str
    password1: str
    password2: str

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()

    @validator("password2")
    def passwords_match(cls, v, values, **kwargs):
        if "password1" in values and v != values["password1"]:
            raise ValueError("passwords do not match")
        return v


def test_does_throw_for_name():

    with pytest.raises(ValidationError):
        UserModel(name="SamuelColvin", password1="zxcvbn", password2="zxcvbn")

def test_no_default_does_throw_for_name():

    with pytest.raises(ValidationError):
        UserModelNoDefaultValue(name="SamuelColvin", password1="zxcvbn", password2="zxcvbn")

def test_should_throw():

    with pytest.raises(ValidationError):
        UserModel(name="Samuel Colvin", password1="zxcvbn", password2="zxcvbn2")

def test_no_default_should_throw():

    with pytest.raises(ValidationError):
        UserModelNoDefaultValue(name="Samuel Colvin", password1="zxcvbn", password2="zxcvbn2")

[dmontagu]

EDIT: I'm retaining this comment for future reference, but it was wrong. See below for resolution.

---

Original comment

1. I agree this is confusing
2. It is currently possible to achieve the validation you want:

class UserModel(BaseModel):
    name: str
    password1: str = "hello"
    password2: str

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()

    @validator("password2")
    def passwords_match(cls, v, values, field):
        password1 = values.get("password1", field.default)
        if v != password1:
            raise ValueError("passwords do not match")
        return v

1. I think modifying the values dictionary to support this application would incur a performance penalty everywhere (even if not making use of the default values).

In general, I agree it would be nice if there were an api that didn't require special casing like this. I also think it would be especially nice if you could use an attribute-access pattern for the autocompletion/refactoring benefits, rather than using a bare dictionary. I'm not sure what the performance implications would be though.

[kognate]

The validation you propose causes a "passwords do not match" error even if the passwords do match. see the test below test_should_not_throw which produces

============================================== FAILURES ===============================================
________________________________________ test_should_not_throw ________________________________________

    def test_should_not_throw():
>       u = UserModel(name="Samuel Colvin", password1="zxcvbn2", password2="zxcvbn2")

prevalid.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pydantic/main.py:275: in pydantic.main.BaseModel.__init__
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   pydantic.error_wrappers.ValidationError: 1 validation error for UserModel
E   password2
E     passwords do not match (type=value_error)

pydantic/main.py:785: ValidationError
===================================== 1 failed, 4 passed in 0.08s =====================================

full code for that here:

from pydantic import BaseModel, ValidationError, validator
import pytest


class UserModel(BaseModel):
    name: str
    password1: str = "hello"
    password2: str

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()


    @validator("password2")
    def passwords_match(cls, v, values, config, field):
        password1 = values.get("password1", field.default)
        if v != password1:
            raise ValueError("passwords do not match")
        return v

class UserModelNoDefaultValue(BaseModel):
    name: str
    password1: str
    password2: str

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()

    @validator("password2")
    def passwords_match(cls, v, values, **kwargs):
        if "password1" in values and v != values["password1"]:
            raise ValueError("passwords do not match")
        return v


def test_does_throw_for_name():

    with pytest.raises(ValidationError):
        UserModel(name="SamuelColvin", password1="zxcvbn", password2="zxcvbn")

def test_no_default_does_throw_for_name():

    with pytest.raises(ValidationError):
        UserModelNoDefaultValue(name="SamuelColvin", password1="zxcvbn", password2="zxcvbn")

def test_should_throw():

    with pytest.raises(ValidationError):
        UserModel(name="Samuel Colvin", password1="zxcvbn", password2="zxcvbn2")

def test_should_not_throw():
    u = UserModel(name="Samuel Colvin", password1="zxcvbn2", password2="zxcvbn2")
    assert u.name == "Samuel Colvin"

def test_no_default_should_throw():

    with pytest.raises(ValidationError):
        UserModelNoDefaultValue(name="Samuel Colvin", password1="zxcvbn", password2="zxcvbn2")

It appears that the value of values in the validator is empty if there is a default value on the field.

[dmontagu]

Okay, that's a real problem. Let me dig in a little.

[dmontagu]

@kognate I got to the bottom of this -- the problem isn't actually what either of us thought is going on.

It is related to this warning in the docs

Warning

Be aware that using annotation only fields will alter the order of your fields in metadata and errors: annotation only fields will always come first, but still in the order they were defined.

I believe this is actually fixed in v1.0, so that only fields without annotations at all come after fields with annotations. (This is closer to a fundamental limitation of python -- as far as we have been able to determine, you can't determine the relative ordering of annotation-only and unannotated fields.)

[dmontagu]

The problem was actually that validators were being executed in the wrong order. Here's a fix:

class UserModel(BaseModel):
    name: str
    password1: str = "hello"
    password2: str = ...

    @validator("name")
    def name_must_contain_space(cls, v):
        if " " not in v:
            raise ValueError("must contain a space")
        return v.title()

    @validator("password2")
    def passwords_match(cls, v, values):
        if v != values["password1"]:
            raise ValueError("passwords do not match")
        return v

Note that it doesn't require you to do anything funny with the default value.

[kognate]

That does fix the issue, thank you.

[dmontagu]

Glad to hear it!