Pydantic: Settings management script advertised as complete, should run "as-is", does not

[metaperl]

The settings management docs provide a script, claiming

(This script is complete, it should run "as is")

But running it yields:

]0;~/prg/tmp
terre@LAPTOP-COM5KPG7 ~/prg/tmp
$ python try_pydantic.py 
Traceback (most recent call last):
  File "try_pydantic.py", line 47, in <module>
    print(Settings().dict())
  File "/usr/lib/python3.7/site-packages/pydantic/env_settings.py", line 34, in __init__
    **__pydantic_self__._build_values(values, _env_file=_env_file, _env_file_encoding=_env_file_encoding)
  File "/usr/lib/python3.7/site-packages/pydantic/main.py", line 346, in __init__
    raise validation_error
pydantic.error_wrappers.ValidationError: 2 validation errors for Settings
auth_key
  field required (type=value_error.missing)
api_key
  field required (type=value_error.missing)
]0;~/prg/tmp
terre@LAPTOP-COM5KPG7 ~/prg/tmp
$ 

Furthermore

How could this line work:

    api_key: str = Field(..., env='my_api_key')

And why is someone new to this package supposed to know what to fill in with the ellipsis?

Suggestion

Make sure the example truly is complete and runs as-is.

[metaperl]

Here's the script, it 's nothing but a copy of what is in the docs:

from typing import Set

from pydantic import (
    BaseModel,
    BaseSettings,
    PyObject,
    RedisDsn,
    PostgresDsn,
    Field,
)


class SubModel(BaseModel):
    foo = 'bar'
    apple = 1


class Settings(BaseSettings):
    auth_key: str
    api_key: str = Field(..., env='my_api_key')

    redis_dsn: RedisDsn = 'redis://user:pass@localhost:6379/1'
    pg_dsn: PostgresDsn = 'postgres://user:pass@localhost:5432/foobar'

    special_function: PyObject = 'math.cos'

    # to override domains:
    # export my_prefix_domains='["foo.com", "bar.com"]'
    domains: Set[str] = set()

    # to override more_settings:
    # export my_prefix_more_settings='{"foo": "x", "apple": 1}'
    more_settings: SubModel = SubModel()

    class Config:
        env_prefix = 'my_prefix_'  # defaults to no prefix, i.e. ""
        fields = {
            'auth_key': {
                'env': 'my_auth_key',
            },
            'redis_dsn': {
                'env': ['service_redis_dsn', 'redis_url']
            }
        }


print(Settings().dict())
"""
{
    'auth_key': 'xxx',
    'api_key': 'xxx',
    'redis_dsn': RedisDsn('redis://user:pass@localhost:6379/1',
scheme='redis', user='user', password='pass', host='localhost',
host_type='int_domain', port='6379', path='/1'),
    'pg_dsn': PostgresDsn('postgres://user:pass@localhost:5432/foobar',
scheme='postgres', user='user', password='pass', host='localhost',
host_type='int_domain', port='5432', path='/foobar'),
    'special_function': <built-in function cos>,
    'domains': set(),
    'more_settings': {'foo': 'bar', 'apple': 1},
}
"""

[samuelcolvin]

Yes, I guess we need to add "provided the environment variables X and Y are set".

PR welcome to fix this.

[StephenBrown2]

Also, the ellipsis: ... isn't meant to be filled in. It's a representation for "no default value" i.e. "this field is required", since Field requires a posarg as it's first parameter. See https://pydantic-docs.helpmanual.io/usage/models/#required-fields
and https://docs.python.org/dev/library/constants.html#Ellipsis and https://docs.python.org/dev/library/stdtypes.html#the-ellipsis-object for it's reference in the Python docs.

[metaperl]

 'env': ['service_redis_dsn', 'redis_url']

also confuses me. Is redis_dsn being set to a list? Or is the list concatenated? Furthermore, how would this parameter be configured as an environmental variable? Via colon-separated values?

[samuelcolvin]

These are different options for the env variable. I believe that is documented. But it's unrelated.

[metaperl]

These are different options for the env variable.

I see. In other words my_prefix_service_redis_dsn or my_prefix_redis_url could be defined as environmental variables to configure this value.

I think my comment above, could be put as a comment right after the code in the example to clarify that. E.g:

 'env': ['service_redis_dsn', 'redis_url']  # These are different options for **naming** the env variable. Be sure to include the prefix.

I believe that is documented.

It is but I think it could be clearer. Here is what I would say:

Settings.Config.fields in the example above specifies custom environment variable names. We see examples of both a string and a list of strings (pecifying a list of options for the name) being provided.

But it's unrelated.

Hmm, I would definitely like to see more connection between the abstract description and the concrete example, perhaps line-by-line through the example, making sure that all abstract aspects of Pydantic-settings is covered in the concrete and fully-working example.