Pydantic: Chaining or merging models

[cdeil]

Question

We are considering using pydantic to handle configuration for our science analysis code.
Currently we have a JSON schema (see here), and a custom class AnalysisConfig (see here) that uses the schema and validates config files the users write in YAML. It's really not well done, there are many issues, and we think pydantic will solve almost all of them, even if it wan't really designed with that use case in mind. Some prototyping is here.

One feature we are missing in pydantic is to allow to have multiple config files and to "chain" or "merge" them, in a way how e.g. https://github.com/Simplistix/configurator allows config to come from multiple places. Currently we have a handwritten method here to do hierarchichal merging of config dicts.

Is there a way to "chain" or "merge" or "update" two pydantic models?

We tried this:

from pydantic import BaseModel

class A(BaseModel):
    s: int = 3

class B(BaseModel):
    x: str = None
    y: str
    a: A

    def update(self, other):
        return self.copy(update=other.dict())

m1 = B(x="hi", y="ho", a=A(s=4))
m2 = B(x="hi2", y="ho2", a=A(s=99))

print(m2.dict())
print(m1.update(m2))
print(m2)

which results in this:

{'x': 'hi2', 'y': 'ho2', 'a': {'s': 99}}
x='hi2' y='ho2' a={'s': 99}
x='hi2' y='ho2' a=A(s=99)

It's not quite what we want, we'd like to have two model instances to represent e.g. "default" and "user-supplied" config, and then to have a method like m1.update(m2) which hierarchically walks the model structure and updates all fields on m1 in-place, for the items in m2 where a non-default value was set (I guess those are all None entries).

Apologies for the very vague question / feature request.

If it's not clear, we can try to come up with a custom method that does what we want and then ask again if it would be in scope as an addition in pydantic or not.

[samuelcolvin]

I don't think this should be particularly difficult, we even have a method to help in utils.

(This is me guessing in the comment box in chrome, not writing something proper in an editor):

from pydantic.utils import deep_update

class A(BaseModel):
    s: int = 3

class B(BaseModel):
    x: str = None
    y: str
    a: A

m1 = B(x="hi", y="ho", a=A(s=4))
m2 = B(x="hi2", y="ho2", a=A(s=99))

m3_dict = deep_update(m1.dict(exclude_defaults=True), m2.dict(exclude_defaults=True))

# if you want to end up with a model, not a dict, then do

m3 = B.construct(**m3_dict)

Let me know whether or not that works.

[Bultako]

Thanks very much for the quick answer.
Your solution does work right as we wish, the minor modif I've done is replacing:
m3 = B.construct(**m3_dict) by m3 = B(**m3_dict)

[remisphere]

I had the same problem trying to merge settings from a configuration file and settings from CLI.
I think this snippet should be in the Settings management section of the documentation.