Hypothesis: Including optional fields in generated examples using the from_type strategy

Created on 24 Oct 2020 · 3Comments · Source: HypothesisWorks/hypothesis

Awesome, awesome project!

The from_type strategy is really useful when generating data from pydantic models (see https://github.com/samuelcolvin/pydantic/issues/1666), but it would be nice if it would recognise optional fields and _sometimes_ generate values for these fields, resulting in a more complete set of examples. The behaviour could be the same as that of the fixed dictionaries strategy when provided with the optional keyword argument, except that the optional fields are already known, from the model type hints.

To give a concrete example:

from typing import Optional

from hypothesis import given
from hypothesis.strategies import from_type
from pydantic import BaseModel

class Adress(BaseModel):
    city: str
    street: str
    house_number: int
    postal_code: int


class Person(BaseModel):
    prename: str
    middlename: Optional[str]
    lastname: str
    address: Adress


@given(from_type(Person))
def test_me(person: Person):
    assert isinstance(person, Person)

It would be great if the behaviour was that the middlename field _may or may not_ be in the generated value. (Currently, it's always None.) Maybe something like:

@given(from_type(Person, include_optional=True))
def test_me(person: Person):
    assert isinstance(person, Person)

Is this a sensible idea, or am I missing some other easy way to do this?

So far I've tried:

Using fixed_dictionaries - not great as you either have to manually provide the required and optional fields and their strategies, or write your own functions to extract them from the pydantic model
@given(st.builds(Person, middlename=infer)) - middlename is always set, which isn't what we want.

bug

Source

garry-jeromson

All 3 comments

st.builds(Person, middlename=infer) is the idiomatic way to do it, possibly with a st.register_type_strategy() to make st.from_type() work too thereafter. Unfortunately it's not working right now, due to some complicated introspection issues below. As a workaround until we fix this, you could use hypothesis_jsonschema.from_schema(Person.schema()).map(Person).

The underlying problem is as follows:

import inspect, typing
from hypothesis.internal.compat import get_type_hints as hypothesis_hints
from pydantic import BaseModel

class Person(BaseModel):
    middlename: typing.Optional[str]

assert typing.get_type_hints(Person) == {'middlename': typing.Union[str, NoneType]}
assert hypothesis_hints(Person) == {'middlename': str, ...}

# and this ultimately comes down to...
assert Person.__annotations__['middlename'] == Optional[str]
assert Person.__signature__.parameters['middlename'].annotation == str

# but consider the following - I think we need to make our __signature__
# logic explicitly handle the default value of None:
def f(a: str=None): ...

assert typing_hints(f) == {'a': typing.Union[str, NoneType]}
assert inspect.signature(f).parameters["a"].annotation == str