Pydantic: Differentiating external and internal data

[notpushkin]

Question

             pydantic version: 1.4
            pydantic compiled: True
                 install path: /home/ale/dev/lunni/lib/.venv/lib/python3.7/site-packages/pydantic
               python version: 3.7.5 (default, Nov 20 2019, 09:21:52)  [GCC 9.2.1 20191008]
                     platform: Linux-5.3.0-46-generic-x86_64-with-Ubuntu-19.10-eoan
     optional deps. installed: []

I'm trying to use Pydantic models as the base “internal” models in my app. It worked fairly well for a while, however I got to a point where I need to do validation when getting the data from outside (e. g. readOnly fields shouldn't be present at all) but not inside the app (it should be present when returning data back to user after all). Something like:

class User(BaseModel):
    id: int = Field(None, readOnly=True)
    username: str

    @validator('id')
    def field_is_readonly(cls, v):
        # There might be a more elegant solution, but for now it'll do
        raise ValueError('field is read-only')

User.load(id=5, username="foo")
# same as User(**dict) as of now
# -> ValidationError: 1 validation error for User
#    id
#      field is read-only (type=value_error)

User.load_trusted(id=5, username="foo")
# e. g. from orm (like in #1212) -- this should be the User(**dict) IMO
# -> User(id=5, username="foo")

I know there's a few issues like this (#422 and #1212, closer to the latter) but I think this is a more fundamental design question than these. I hope I didn't get too metaphysical and still do make some sense :) Feel free to ask me to clarify if I don't.

Bonus: hacky writeOnly implementation

from pydantic import BaseModel, Field

EMPTY_SET = set()

def get_write_only_fields(schema):
    if schema.get("writeOnly", False):
        return ...
    elif schema.get("type") == "object":
        return {
            name: get_write_only_fields(inner_schema)
            for name, inner_schema in schema["properties"].items()
        }

    return EMPTY_SET


class Model(BaseModel):
    def dict(self, **kwargs):
        kwargs.setdefault("exclude", get_write_only_fields(self.schema()))
        return super().dict(**kwargs)

This would completely hide SecretStr / SecretBytes and other writeOnly values from model.dict() but retain them in dict(model) and attributes.

[samuelcolvin]

I think your solution is fine.

Long term I suspect we should extend load and dump aliases #624 to include a "skip" option to exclude things from from dict() and json().