Pydantic: add exclude to Config

[samuelcolvin]

From a user perspective I would rather add an exclude in the Config metaclass instead of passing the dict in to .json(). One of my model's fields is a Callable and I would like to call .json() on it, however I need to instead pass a custom encoder to the .json() which still puts the responsibility on the caller.

_Originally posted by @enkoder in https://github.com/samuelcolvin/pydantic/pull/648#issuecomment-511265737_

[samuelcolvin]

as per #624 we could easily add exclude to a field, or even a dict to exclude sub-fields.

[JrooTJunior]

I guess to implements this like exporting roles in Schematics
That mechanism is good for writing API's

You can simply describe all of public fields in model and inside controllers make dump in required set of fields by specifying only the role name.

For example:

class User(BaseModel):
    id: int
    login: str
    email: str
    password: str
    first_name: str
    last_name: Optional[str] = None
    articles: Optional[List[Article]] = None

    register_date: datetime.datetime
    last_login: Optional[datetime.datetime] = None

    class Config:
        roles = {
            "self": {"exclude": {"password", "register_date", "last_login"}},
            "admin": {"exclude": {"password"}},
            "public": {"include": {"id", "email", "first_name", "last_name"}}
        }

And then:

user = User(
    id=42,
    login="mylogin",
    email="[email protected]",
    password="hashed secret password",
    first_name="Some",
    last_name="User",
    articles=[Article(id=1, label="foo"), Article(id=2, label="bar"), ...],
    register_date=datetime.datetime(...),
    last_login=datetime.datetime.now()
)
user.dump(role="public")

Should return something like this:

{
    "id": 42,
    "email": "[email protected]",
    "first_name": "Some",
    "last_name": "User",
}

include/exclude describing mechanism can be implemented more simply, for e.g. via function include(...)/exclude(...)

[enkoder]

Wow thanks for the quick response @samuelcolvin. I have just been exploring pydantic and I really like it. Having quick responses on PR's and active development certainly makes me even more excited to adopt it. Looking forward to working with you.

[dgasmith]

+1 for this, we code it into our classes as we have "local" variables that should not exported to other objects. Allowing this in the dict() on up would be optimal.

[MrMrRobat]

I like this idea and love @JrooTJunior's implementation with roles
I guess I'll be able to work on a new PR with these changes very soon, right after #648 will be merged

[samuelcolvin]

I understand that roles is neat, but I think it's a whole new concept that is unnecessarily complex for pydantic. If you did want roles it would be fairly trivial to add it to config and implement your own CustomBaseModel that used it as per #684.

I think exclude and include should use the fields property already in Config and also be configurable on Schema (soon to be Field, #577).

Combining this with #624, we'd have something like:

class User(BaseModel):
    id: int
    username: str
    password: str

class Transaction(BaseModel):
    id: str
    user: User
    value: int

    class Config:
        fields = {
            'value': {
                'alias': 'Amount',
                'exclude': ...,
            },
            'user': {
                'exclude': {'username', 'password'}
            },
            'id': {
                'dump_alias': 'external_id'
            }
        }

[xspirus]

Hello @samuelcolvin . First of all, I am really amazed by pydantic and, even though I haven't been using it for a long time, I think it can be really helpful in numerous situations.

On the point of this issue, as I understand, this has not been implemented yet, correct? Is there currently any work going on for this? If not, I am willing to have a go on this. Having include and exclude on Field and Config, in the way you described above, sounds like a really nice feature.

[mtucker502]

I solved this today with the following:

class Resource(BaseModel):
    class Config:
        include: Set[str] = set()
        exclude: Set[str] = set()

    def json(self, **kwargs):
        include = kwargs.pop('include', set())
        include = include.union(getattr(self.Config, "include", set()))
        if len(include) == 0:
            include = None

        exclude = kwargs.pop('exclude', set())
        exclude = exclude.union(getattr(self.Config, "exclude", set()))
        if len(exclude) == 0:
            exclude = None

        return super().json(include=include, exclude=exclude, **kwargs)

Just specify the include/exclude fields as a part of the Class definition:

class SR(Resource):
    synopsis: str
    session: str

    class Config:
        exclude = {'session'}

[xspirus]

@mtucker502 That is one way to solve when calling the .json method. However, I think that this issue tries to solve this kind of problem:

class User(BaseModel):
    id: int
    username: str
    password: str

class Transaction(BaseModel):
    id: str
    user: User
    value: int

    class Config:
        fields = {
            'user': {
                'exclude': {'username', 'password'}
            },
        }

And then calling the constructor would look like this.

>>> Transaction(id=1, user={"id": 1}, value=1)
Transaction(id=1, user=User(id=1), value=1)

Correct me if I'm wrong @samuelcolvin .

[MrMrRobat]

@xspirus, this is an interesting concept. And it would actually solve/alleviate problem with copiyng the same model with little changes for different contexts.

@enkoder, since this issue is originaly created from your https://github.com/samuelcolvin/pydantic/pull/648#issuecomment-511265737, what do you think?

[samuelcolvin]

Hello @samuelcolvin . First of all, I am really amazed by pydantic and, even though I haven't been using it for a long time, I think it can be really helpful in numerous situations.

On the point of this issue, as I understand, this has not been implemented yet, correct? Is there currently any work going on for this? If not, I am willing to have a go on this. Having include and exclude on Field and Config, in the way you described above, sounds like a really nice feature.

No work on this AFAIK, happy to review a PR if it matches my suggestion above.

[xspirus]

No work on this AFAIK, happy to review a PR if it matches my suggestion above.

I will try to create a PR, as soon as I find some time to work on this :smile:

[enkoder]

@MrMrRobat the approach that @mtucker502 took is how I would expect things to look! Defining excluded and included on the config like they did.

[daviskirk]

I'd be using this as well (or a variant of it) so I can also try and take a swing at a PR.