Pydantic: Models referencing each other from separate files

[dmontagu]

Question

Is there a good way to have multiple models reference each other from separate files?

For example:

# models/a_model.py
from typing import Optional
from pydantic import BaseModel
from models.b_model import BModel

class AModel(BaseModel):
    b: Optional["BModel"]

# models/b_model.py
from typing import Optional
from pydantic import BaseModel
from models.a_model import AModel

class BModel(BaseModel):
    a: Optional["AModel"]

If I put both in the same file it just works, but I'm trying to make openapi-generator templates with pydantic models as the base models, and typically each model goes in its own file. I can't figure out if there is a way around the import issues.

[samuelcolvin]

just use a: Optional[AModel] it's just a standard python import, nothing specific to pydantic.

[dmontagu]

@samuelcolvin You are right, it is not specific to pydantic, but I figured other users of pydantic may be aware of a workaround for the problem (specifically because I think the one-class-per-file-with-lots-of-cross-references pattern may be more common when working with something like BaseModel/dataclass/etc.).

I tried with quotes removed too, it doesn't make a difference in the error output. Trying to run the files above exactly as written, with or without the quotes (also the result is the same if I comment out the pydantic import and don't inherit from BaseModel):

Traceback (most recent call last):
  File "models/a_model.py", line 4, in <module>
    from models.b_model import BModel
  File "models/b_model.py", line 4, in <module>
    from models.a_model import AModel
  File "models/a_model.py", line 4, in <module>
    from models.b_model import BModel
ImportError: cannot import name 'BModel' from 'models.b_model'

This is using Python 3.7.3.

As it turns out, I was able to work around this problem for client generation by just auto-merging all of the model files into one; I think this is more or less acceptable for the use case, so I don't think a better solution is necessary. But it would be nice if handling the cyclic imports worked a little better.

[ri-gilfanov]

@samuelcolvin, a little comparison of dataclasses.dataclass and pydantic.BaseModel behavior.

Simple data classes

my_package/init.py:

import module_a
import module_b

my_package/module_a.py:

from dataclasses import dataclass
from typing import Optional
import my_package


@dataclass
class ClassA:
    attr_b: Optional['my_package.module_b.ClassB']

my_package/module_a.py:

from dataclasses import dataclass
from typing import Optional
import my_package


@dataclass
class ClassB:
    attr_a: Optional['my_package.module_a.ClassA']

my_package/module_c.py:

from inspect import signature
from my_package.module_a import ClassA
from my_package.module_b import ClassB


print(signature(ClassA.__init__))
print(signature(ClassB.__init__))

obj_a = ClassA(attr_b=None)
obj_b = ClassB(attr_a=None)

print(obj_a)
print(obj_b)

Run my_package/module_c.py and look at the output:

(self, attr_b: Union[ForwardRef('my_package.module_b.ClassB'), NoneType]) -> None
(self, attr_a: Union[ForwardRef('my_package.module_a.ClassA'), NoneType]) -> None
ClassA(attr_b=None)
ClassB(attr_a=None)

It works.

BaseModel from Pydantic

Change my_package/module_a.py:

from pydantic import BaseModel
from typing import Optional
import my_package


class ClassA(BaseModel):
    attr_b: Optional['my_package.module_b.ClassB']

Change my_package/module_a.py:

from pydantic import BaseModel
from typing import Optional
import my_package


class ClassB(BaseModel):
    attr_a: Optional['my_package.module_a.ClassA']

Try to run my_package/module_c.py and look at the output:

Traceback (most recent call last):
  File "~/projects/my_package/module_c.py", line 4, in <module>
    from my_package.module_a import ClassA
  File "~/projects/my_package/__init__.py", line 1, in <module>
    import module_a
  File "~/projects/my_package/module_a.py", line 7, in <module>
    class ClassA(BaseModel):
  File "pydantic/main.py", line 237, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/typing.py", line 161, in pydantic.typing.resolve_annotations
    if isinstance(obj, type):
  File "/usr/lib/python3.8/typing.py", line 272, in _eval_type
    ev_args = tuple(_eval_type(a, globalns, localns) for a in t.__args__)
  File "/usr/lib/python3.8/typing.py", line 272, in <genexpr>
    ev_args = tuple(_eval_type(a, globalns, localns) for a in t.__args__)
  File "/usr/lib/python3.8/typing.py", line 270, in _eval_type
    return t._evaluate(globalns, localns)
  File "/usr/lib/python3.8/typing.py", line 518, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
AttributeError: partially initialized module 'my_package' has no attribute 'module_b' (most likely due to a circular import)

Something went wrong. The exception is raised on model import.

Why this comparison

I found a similar case when I tried to implement models for the Activity Pub protocol using Pydantic. There the core type "Object" must have an attribute of the child type "Image". Everything works in one module, but I would like to keep the core and child types in separate modules.

I think the above difference in behavior is more like a bug. This behavior complicates the simple replacement of data classes with Pydantic models and narrows the applicability Pydantic in a number of recurring cases. For example, implementation of generally accepted Internet protocols by specification, with division into modules and validation on type annotations.

Thanks.