Pydantic: [Question] How to parse partial list of nested pydantic entities?

[simkimsia]

pydantic==1.5.1

I have the following

class DataLineItem(BaseModel)
    RT_Request_ID: int
    PR_Creation_Date: datetime
    PR_Number: str


class DataList(BaseModel):
    # need the alias or else fields that start with underscore will be ignored
    # see https://github.com/samuelcolvin/pydantic/issues/1496#issuecomment-626443365
    items: conlist(PRPODataLineItemFromPDB, min_items=1) = Field(alias="_items")

then I do a parse_obj on a json object

p = DataList.parse_obj(expected_json)

my question is what if the expected_json is like this

expected_json = {"_items": [
  {"RT_Request_ID": 123, "PR_Creation_Date": "2020-01-01 01:01:12", "PR_Number":"abc"},
  {"RT_Request_ID": 124, "PR_Creation_Date": None, "PR_Number":"abc"}
]}

Obviously the 2nd line item will fail the validation since "PR_Creation_Date" is compulsory.

Is there a way I can parse the json object but not trigger validation error and simply just get back the parsed items to be just 1 valid item?

[StephenBrown2]

Using a pre-validator sounds like the perfect way to handle this:

from datetime import datetime

from pydantic import BaseModel, Field, ValidationError, conlist, validator

expected_json = {
    "_items": [
        {"RT_Request_ID": 123, "PR_Creation_Date": "2020-01-01 01:01:12", "PR_Number": "abc"},
        {"RT_Request_ID": 124, "PR_Creation_Date": None, "PR_Number": "abc"},
    ]
}


class DataLineItem(BaseModel):
    RT_Request_ID: int
    PR_Creation_Date: datetime
    PR_Number: str


class DataList(BaseModel):
    items: conlist(DataLineItem, min_items=1) = Field(alias="_items")

    @validator("items", pre=True)
    def only_valid_items(cls, v):
        valid_items = []
        for item in v:
            try:
                DataLineItem(**item)
            except ValidationError:
                pass
            else:
                valid_items.append(item)

        if not valid_items:
            raise ValueError("No valid items provided")

        return valid_items

p = DataList.parse_obj(expected_json)
len(p.items) == 1

[simkimsia]

Very grateful to your answer, @StephenBrown2 🙏

I made a small change

      for item in v:
            try:
                # only attempt to convert if it’s not already the nested pydantic entity
                if isinstance(item, DataLineItem) is False:
                    DataLineItem(**item)
            except ValidationError:
                pass
            else:
                valid_items.append(item)

So that it works with both

• .parse_obj and
• regular assignment like:

p_list = DataList(_items=[p1, p2]) # where p1 and p2 are already DataLineItem