Pydantic: What's the Preferred Way to Derive One Setting From Another?

[eceppda]

Question

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.5.1
            pydantic compiled: True
                 install path:[...]/venv/lib/python3.7/site-packages/pydantic
               python version: 3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC 7.3.0]
                     platform: Linux-5.4.0-26-generic-x86_64-with-debian-bullseye-sid
     optional deps. installed: []

I'd like to derive one setting from another, like the following:

from pydantic import BaseSettings, DirectoryPath, FilePath


class Settings(BaseSettings):

    root_dir: DirectoryPath = '/foo/bar'

    file_one: FilePath = f'{root_dir}/baz.txt'

    file_two: FilePath = f'{root_dir}/baz.md'

The issue here is, if I override root_dir with a different environment value, the derived properties remain set with the default values. I roughly understand that this is because the defaults are applied when the class is initialized and cannot be updated later.

What is the preferred way to derive settings?

I have implemented the following, but I lose BaseModel validation for file_one:

from pydantic import BaseSettings, DirectoryPath, FilePath


class Settings(BaseSettings):

    root_dir: DirectoryPath = '/foo/bar'

    @property
    def file_one(self) -> FilePath:
        return f'{self.root_dir}/baz.txt'

Is there a better way to do this?

[samuelcolvin]

you should be able to use a validator with pre=True.

Because the root_dir field is defined first, you can use it (via the values argument to the validator) to get root_dir. Because this is a pre validator, the standard validation will happen after you've created the path/str.

[eceppda]

Thank you for your response!

I implemented the following based on this recommendation:

import os
from pathlib import Path

from pydantic import BaseSettings, DirectoryPath, FilePath, validator, errors


class Settings(BaseSettings):

    root_dir: DirectoryPath = '/foo/bar'

    file_one: FilePath = f'{root_dir}/baz.txt'

    file_two: FilePath = f'{root_dir}/baz.md'

    @validator('root_dir', pre=True)
    def validate_root_dir(cls, value):

        if os.environ.get('root_dir'):
            value = os.environ.get('root_dir')

        if not Path(value).is_dir():
            raise errors.PathNotADirectoryError(path=value)

        return value

I validate either the default or environment value of root_dir, and set the validator to pre=True to evaluate before the other settings.

My test is to export root_dir=/valid/path and then initialize Settings class in a Python interpreter.

>>> from test import Settings
>>> settings = Settings()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pydantic/env_settings.py", line 28, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic/main.py", line 338, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 2 validation errors for Settings
file_one
  path "/foo/bar/baz.txt" does not point to a file (type=value_error.path.not_a_file; path=/foo/bar/baz.txt)
file_two
  file or directory at path "/foo/bar/baz.md" does not exist (type=value_error.path.not_exists; path=/foo/bar/baz.md)

You can see that the settingsfile_one & file_two, still used the initialized value of root_dir, and not the updated value as I had hoped.

[samuelcolvin]

That's not what I meant, here (with funky python 3.8 syntax you can remove if you like) is what I meant:

from pydantic import BaseSettings, DirectoryPath, FilePath, validator
from devtools import debug


class Settings(BaseSettings):
    root_dir: DirectoryPath = '/foo/bar'

    file_one: FilePath = 'baz.txt'
    file_two: FilePath = 'baz.md'

    @validator('file_one', 'file_two', pre=True)
    def apply_root(cls, v, values):
        if root_dir := values.get('root_dir'):
            return root_dir / v
        else:
            # should only happen when there was an error with root_dir
            return v

debug(Settings())