Pydantic: Generic type is not passed into items of a List

[radeklat]

Bug

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

pydantic version: 1.5.1
pydantic compiled: True
install path: /home/.../.venv/lib/python3.8/site-packages/pydantic
python version: 3.8.2 (default, Apr 22 2020, 13:26:47)  [GCC 7.5.0]
platform: Linux-5.3.0-45-generic-x86_64-with-glibc2.27
optional deps. installed: ['typing-extensions']

Using a modified example from the documentation on Generic Models:

from typing import Generic, TypeVar, List
from pydantic.generics import GenericModel

T = TypeVar('T')

class InnerT(GenericModel, Generic[T]):
    inner: T

class OuterT(GenericModel, Generic[T]):
    outer: T
    nested: List[InnerT[T]]

print(OuterT[int](**{"outer": 1, "nested": [{"inner": "2"}]}))  # Output 1
print(OuterT[int](**{"outer": 1, "nested": [{"inner": "a"}]}))  # Output 2

Expected output

1. outer=1 nested=[InnerT[int](inner='2')]
2. Validation error

Actual output

1. outer=1 nested=[InnerT[T](inner='2')]
2. outer=1 nested=[InnerT[T](inner='a')]

[gregbrowndev]

I have a very similar use case but involving specifying bound on the type parameter. However, this results in a pydantic.errors.ConfigError:

pydantic.errors.ConfigError: field "dataset" not yet prepared so type is still a ForwardRef, you might need to call Revision[T].update_forward_refs().

I'm not sure if this is a related bug or something that I'm doing wrong.

from pydantic import BaseModel
from pydantic.generics import GenericModel

T = TypeVar("T", bound="Dataset")


class Dataset(BaseModel):
    name: str
    description: str


class Revision(GenericModel, Generic[T]):
    publication_id: PublicationId
    dataset: T


class Publication(GenericModel, Generic[T]):
    id: int
    revisions: List[Revision[T]]


if __name__ == '__main__':
    pub = Publication[Dataset](
        id=1,
        revisions=[
            Revision[Dataset](
                publication_id=1, dataset=Dataset(name="XYZ", description="...")
            )
        ],
    )
    print(pub)

Fortunately, removing bound allows it to work but I do see the same issue as above:

print(pub)
# id=1 revisions=[Revision[T](publication_id=1, dataset=Dataset(name='XYZ', description='...'))]

[StephenBrown2]

@gregbrowndev Have you tried following the instructions in the error message?

you might need to call Revision[T].update_forward_refs()

Doing so allows me to run your code successfully including bound:

from typing import Generic, List, TypeVar

from devtools import debug

from pydantic import BaseModel
from pydantic.generics import GenericModel

T = TypeVar("T", bound="Dataset")


class Dataset(BaseModel):
    name: str
    description: str


class Revision(GenericModel, Generic[T]):
    publication_id: PublicationId
    dataset: T


class Publication(GenericModel, Generic[T]):
    id: int
    revisions: List[Revision[T]]

Revision[T].update_forward_refs()

pub = Publication[Dataset](
    id=1,
    revisions=[
        Revision[Dataset](
            publication_id=1, dataset=Dataset(name="XYZ", description="...")
        )
    ],
)
debug(pub)
# Outputs:
    pub: Publication[Dataset](
        id=1,
        revisions=[
            Revision[T](
                publication_id=1,
                dataset=Dataset(
                    name='XYZ',
                    description='...',
                ),
            ),
        ],
    ) (Publication[Dataset])

[gregbrowndev]

@StephenBrown2 thanks a lot, that works for me too!

I did try calling update_forward_refs in a number of ways but was thrown off by the [T] :facepalm:

[StephenBrown2]

I'm not terribly familiar with Generics either, but if it works I'm glad!

[samuelcolvin]

Thanks @StephenBrown2 for answering @gregbrowndev's question, I'v marked those comments as off-topic to keep this discussion targeted.

---

@radeklat

I can replicate the error. To be clear both nested: List[T] and nested: InnerT[T] work, just not nested: List[InnerT[T]].

I have no idea how hard this would be to fix, happy to review a PR.

[amartinunowhy]

Hello,

Thank you for this great library. The following commit is a solution for this issue:

https://github.com/amartinunowhy/pydantic/commit/8d953ca389131f705e27bca37097ed59081dda7e

Basically the problem is that the current code only looks for GenericModel types to _concretize_. The fix makes it recurse into List, Union and Tuple types. The problem for me was with Optional[DataT], which is actually Union[DataT, None].

I'm not making a PR yet as I believe the code is only 3.8 compliant and some tests need to be written. I'm don't know when I will have the bandwidth to do that. Sorry.

It doesn't break any current test though.