Fastapi: [FEATURE] Inheritance and Polymorphism support

[tchiotludo]

Actually, there is no support for Inheritance and Polymorphism as I know in fastapi : https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/

Maybe annotations like that will do the job :

class Item(BaseModel):
    type: str

class CarItem(Item):
    type: str

class PlaneItem(Item):
    type: str
    size: str


@app.post("/items/", response_model=Item, one_of=[CarItem, PlaneItem], discriminator="type")
async def create_item(*, item: Item):
    return item

For now, we need to define an any response_model since defining an Item response_model will remove the extra attributes during serialization.

I'll try to hack using the based starlette api : https://www.starlette.io/schemas/ but comment are unused by fastapi (except for description)

[tiangolo]

You can achieve what I think you need with standard Python typing.Union.

I just added docs documenting it: https://fastapi.tiangolo.com/tutorial/extra-models/#union-or-anyof

[tchiotludo]

Thanks for the response !!!
It works well on response thanks !! :+1:

But I also try with request body :

class Item(BaseModel):
    type: str

class CarItem(Item):
    type: str
    wheel: int

class PlaneItem(Item):
    type: str
    size: str


item_union = Union[
    CarItem,
    PlaneItem
]

@app.post("/items/", response_model=item_union)
async def create_item(*, item: item_union):
    return item

seems to pick the first one, so drop the fields that is not on the second one.

Any suggestion for this one ?

[euri10]

confirmed with a set of tests, specifically test_create_item_reverse: https://github.com/euri10/fastapi/tree/post_union
no time for looking further though ! maybe later :_)
Union order seems important

[euri10]

After some digging, it seems like it comes from upstream

        if self.sub_fields:
            errors = []
            for field in self.sub_fields:
                value, error = field.validate(v, values, loc=loc, cls=cls)
                if error:
                    errors.append(error)
                else:
                    return value, None
            return v, errors

As soon as one sub_fields of the Union is valid it returns.

So the CarItem in my test returns first as it's validated 1st.

https://github.com/tiangolo/fastapi/blob/5a6e47bd495b7ee0bb2f4eb6dce570686c21c554/fastapi/dependencies/utils.py#L358

No idea if that's something to report upstream, that seems logic to consider valid a union as soon as one of its members is valid

Maybe FastAPI should, if there are subfields, loop orderly on them and validate the body from there.

[tchiotludo]

I think this case must be handle by fastapi.

Since pydantic seems to be "compliant" with jsonchema and jsonschema don't support true polymorphism : anyOf, allOf, but no discriminator like OpenApi support.

It seems to be more OpenApi feature than JsonSchema feature IMO.

[tiangolo]

So, in Pydantic, the order of Unions is important, the first thing that can be cast to a type and be valid is used.

Nevertheless, I tested it with your example and it seems to work well (as I expected it).

When I send:

{"type": "plane", "size": "big"}

I receive that back.

And when I send:

{"type": "car", "wheel": 3}

I receive that back too.

What seems to be the problem there?

seems to pick the first one, so drop the fields that is not on the second one.

Maybe I'm not understanding this phrase well? Can you elaborate?

[tchiotludo]

You are right ! I used a simplify example for the issue but this one works like that !
The first win like you said and in my case, the first doesn't have more property.
So it win, and drop all the field for other element.
Just changing the order of union works well.

I little bit tricky, but it works

class Item(BaseModel):
    type: str

class CarItem(Item):
    pass

class PlaneItem(Item):
    plane: int

class TruckItem(Item):
    truck: int


item_union = Union[
    CarItem,
    PlaneItem,
    TruckItem
]


class Container(BaseModel):
    collec: List[item_union]



@app.post("/items/", response_model=Container)
async def create_item(*, item: Container):
    return item

[tiangolo]

Great!

[euri10]

it fails on my test branch on the test_create_item_reverse test, but maybe that's not fixable or normal:

it passes plane = PlaneItem(description="plane description", type="plane", size=10) to the items_reverse route

that route has the beow response model:

@app.post("/items_reverse/", response_model=Union[CarItem, PlaneItem])
async def create_item(*, item: Union[CarItem, PlaneItem]):
    return item

you get effectively a plane back, but the size disappeared in the response as the response_model thought it was a car.

[tchiotludo]

I've dig my code today and I understand my problem :

class Item(BaseModel):
    type: str

class CarItem(Item):
    car: Optional[int]

class PlaneItem(Item):
    plane: Optional[int]

class TruckItem(Item):
    truck: Optional[int]


item_union = Union[
    CarItem,
    PlaneItem,
    TruckItem
]


class Container(BaseModel):
    collec: List[item_union]


@app.post("/items/", response_model=Container)
async def create_item(*, item: Container):
    return item

All the fields are optional, so the first always win.

First one works well :

POST {{host}}/items/
{
  "collec": [
    {
      "type": "plane",
      "plane": 3
    }
  ]
}

But this one :

POST {{host}}/items/
{
  "collec": [
    {
      "type": "truck",
      "plane": null
    }
  ]
}

return :

{
  "collec": [
    {
      "type": "truck",
      "plane": null
    }
  ]
}

In this case of schema, I don't have any solution. It's a normal behavior for my app to have some optional field. Pydantic will always return the first one since it's valid and drop field for the second one.

Now, I have to change the code to make the abstract class always win and keep fields :

class Item(BaseModel):
    type: str

    class Config:
        extra = Extra.allow

item_union = Union[
    Item,
    CarItem,
    PlaneItem,
    TruckItem
]

The response will have extra fields, and I recreate the correct object on my controller.

[tiangolo]

Cool, I think you found a good solution for your use case.

[tiangolo]

I'll close this issue now, but feel free to add more comments or create new issues.

[sm-Fifteen]

@tiangolo: What about the lack of a discriminator field? The doc for Pydantic says they support arbitrary schema parameters, which get added as-is in the schema, so couldn't discriminators be supported this way, possibly?

EDIT: Looks like Pydantic uses anyOf for unions instead of oneOf, so I guess that wouldn't work.

[Rapiz1]

I think real inheritance and polymorphism support would be very useful. Comments above show that using typing.Union forces users to carefully put the class in order and is not a generic solution.

[sm-Fifteen]

I think real inheritance and polymorphism support would be very useful. Comments above show that using typing.Union forces users to carefully put the class in order and is not a generic solution.

The problem is just how schema validation works. With the example given in the original post, a CarItem is always a valid subset of a PlaneItem, there's no way for Pydantic to unambiguously validate an incoming object as being a car without being a plane. That could be avoided by making type be something like a typing.Literal['car'] or typing.Literal['plane'] to give pydantic something to tell both types appart.

"Real" inheritance and polymorphism is not really a concept in OpenAPI and JSON Schema validation (which is what FastAPI relies on to parse JSON into objects), since the data you receive is just an arbitrary JSON object, and the type of that object can only be determined by validating it against object schemas. There's just not much else to do when your object happens to match more than one.

[tchiotludo]

Polymorphism is part of openapi : https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/

The discriminator is the solution that may be implemented on fastapi to have strong validation.

We have succeed to have polymorphism in a fastapi application but with a lot of tricky things that would be simpler with support on webserver.

[sm-Fifteen]

For what it's worth, I've raised an issue upstream (samuelcolvin/pydantic#619) regarding making discriminator available for schemas, but expressing it in an ergonomic way is not trivial. Despite what the spec says, discriminator is meant for representation of tagged unions, not actual inheritance (it just short-circuits the normal validation process for schema selection based on the mapping parameter OR the names of each sub-schema, and that last bit is really annoying to deal with), and Python's typing module doesn't really have a type like that, so one would need to be created.

Support for discriminator in the OpenAPI ecosystem is not great either, with Swagger UI not even supporting it yet (swagger-api/swagger-ui#2438). ReDoc supports it, but I've read that their implementation might be a bit quirky.

I'm hoping discriminator becomes supported by Pydantic/FastAPI eventually, don't get me wrong, but I've come to acknowledge that it's actually really weirdly defined, and I understand why there's talk of replacing it with something more straightforward in the OpenAPI spec eventually.

[Alphasite]

Support for it actually landed in the openapi-generator project for typescript, go, python etc this past release. So there is movement in the right direction.

[mcauto]

Hi! @tchiotludo

from typing import Union

from fastapi.applications import FastAPI
from pydantic import BaseModel


class Human(BaseModel):
    age: int
    name: str


class Man(Human):
    """ man """

    something: str


class Woman(Human):
    """ woman """


class ManCreateRequest(Man):
    class Config:
        schema_extra = {
            "example": {
                "target": Man(age=29, name="deo", something="something").dict()
            }
        }


class HumanCreateRequest(BaseModel):
    target: Union[Woman, Man]

    class Config:
        schema_extra = Man.Config.schema_extra # default Schema Man


app = FastAPI()


@app.post("/human")
def add_human(create_request: HumanCreateRequest) -> None:
    instance_type = create_request.target # always Woman
    return

Pydantic will always return the first one...

I want to get real type of HumanCreateRequest.

[sm-Fifteen]

@mcauto: Ah, that's a common mistake people make. Pydantic can't tell what data structure your JSON is supposed to be aside from matching it against one of your possible models. As it happens, since Woman adds no new fields to Human and Man does, there's no way for Pydantic to tell whether the payload you sent it just a Woman object (essentially a Human object) with extra fields or a Man object, since both match.

If you really want the absence of something to define whether or not your Human subtype is Man or Woman, I believe you can just have Woman forbid the presence of extra fields, like so:

class Woman(Human):
    """ woman """
    class Config:
        extra = 'forbid'

(I have not tested it myself, though, so apologies if this doesn't work)

[mcauto]

@mcauto: Ah, that's a common mistake people make. Pydantic can't tell what data structure your JSON is supposed to be aside from matching it against one of your possible models. As it happens, since Woman adds no new fields to Human and Man does, there's no way for Pydantic to tell whether the payload you sent it just a Woman object (essentially a Human object) with extra fields or a Man object, since both match.

If you really want the absence of something to define whether or not your Human subtype is Man or Woman, I believe you can just have Woman forbid the presence of extra fields, like so:

class Woman(Human):
    """ woman """
    class Config:
        extra = 'forbid'

(I have not tested it myself, though, so apologies if this doesn't work)

@sm-Fifteen Thank you for the kind explanation!

I solved it!

from typing import Union

from fastapi.applications import FastAPI
from pydantic import BaseModel
from pydantic.main import Extra


class Human(BaseModel):
    age: int
    name: str


class Man(Human):
    """ man """

    man_something: str


class Woman(Human):
    """ woman """

    woman_something: str


class ManCreateRequest(Man):
    class Config:
        extra = Extra.forbid
        schema_extra = {
            "example": {
                "target": Man(age=29, name="deo", man_something="man").dict()
            }
        }


class WomanCreateRequest(Woman):
    class Config:
        extra = Extra.forbid
        schema_extra = {
            "example": {
                "target": Woman(
                    age=29, name="woorr", woman_something="woman"
                ).dict()
            }
        }


class HumanCreateRequest(BaseModel):
    target: Union[WomanCreateRequest, ManCreateRequest]

    class Config:
        # schema_extra = ManCreateRequest.Config.schema_extra
        schema_extra = WomanCreateRequest.Config.schema_extra


app = FastAPI()


@app.post("/human")
def add_human(create_request: HumanCreateRequest) -> str:
    if isinstance(create_request.target, WomanCreateRequest):
        return "woman"
    elif isinstance(create_request.target, ManCreateRequest):
        return "man"
    else:
        raise NotImplementedError()