Datamodel-code-generator: oneOf items are not parsed recursively

[m-kus]

Describe the bug
Given the type Union[int, List[A], B]:

• A is treated as Any
• B class is generated empty with extra=allowed

To Reproduce

Example schema:

{"$schema":"http://json-schema.org/draft/2019-09/schema#", "oneOf":[{"type":"integer","$comment":"int"},{"type":"array","items":{"type":"object","properties":{"key":{"type":"object","properties":{"address":{"type":"string","$comment":"address"},"nat":{"type":"string","$comment":"nat"}},"required":["address","nat"],"additionalProperties":false,"$comment":"pair"},"value":{"type":"string","$comment":"nat"}},"required":["key","value"],"additionalProperties":false}}],"$comment":"big_map"}

Used commandline:

$ datamodel-codegen --input schema.json --output schema.py

Expected behavior
All the oneOf sub-schemas are parsed recursively.

Version:

• OS: Ubuntu 18
• Python version: 3.9
• datamodel-code-generator version: 0.10.2

[koxudaxi]

@m-kus
Thank you for creating this issue.
I have confirmed this problem.
I will fix it.

[kmorris81]

+1 on this.

[koxudaxi]

@m-kus @kmorris81
I have released a fixed version as 0.10.3.
I believe the code-generator can parse complex nested stracture.

[kmorris81]

Hi @koxudaxi
Thanks for the code update. I tried it out against this schema and did not see any difference between the previous release (0.10.2)

https://www.liquid-technologies.com/Reference/XmlStudio/JsonEditorNotation_XXXOf.html

I guess I was expecting to see pydantic models generated for the state/zipcode and county/postcode options listed under the oneOf section.

Thoughts?

Thanks again!

[m-kus]

Hey @koxudaxi, my case was fixed in 0.10.3, thanks a lot!
@kmorris81 I think this is a separate case with "factoring" out common parts of the subschemas.

[m-kus]

@koxudaxi was able to reproduce on another sample, here is the snippet:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "oneOf": [
        {
            "type": "integer",
            "$comment": "int"
        },
        {
            "type": "object",
            "propertyNames": {
                "type": "string",
                "$comment": "string"
            },
            "additionalProperties": {
                "type": "string",
                "$comment": "bytes"
            }
        }
    ]
}

The output is

class Model(BaseModel):
    __root__: Union[int, Dict[str, Any]]

which is not what expected

[koxudaxi]

@kmorris81
Thank you for testing the new version.
I guess the reason is other.
I call it is combine model.
a related issue is https://github.com/koxudaxi/datamodel-code-generator/issues/399
I will implement it.

[koxudaxi]

@m-kus
Thank you for the additional comment.

which is not what expected

I don't know propertyNames.
Do you have an idea to write propertyNames in pydnatic?

[m-kus]

I think you can ignore propertyNames as it's always at least {"type": "string"}.
I meant that if additionalProperties type was treated as a fallback value type, we get Dict[str, str] which is more precise.

Here is another case that demonstrates it better:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "oneOf": [
                {
                    "type": "integer",
                    "$comment": "int"
                },
                {
                    "type": "object",
                    "propertyNames": {
                        "type": "string",
                        "$comment": "nat"
                    },
                    "additionalProperties": {
                        "type": "object",
                        "properties": {
                            "token_id": {
                                "type": "string",
                                "$comment": "nat"
                            },
                            "token_info": {
                                "type": "object",
                                "propertyNames": {
                                    "type": "string",
                                    "$comment": "string"
                                },
                                "additionalProperties": {
                                    "type": "string",
                                    "$comment": "bytes"
                                },
                                "$comment": "map"
                            }
                        },
                        "required": [
                            "token_id",
                            "token_info"
                        ],
                        "additionalProperties": false,
                        "$comment": "pair"
                    }
                }
            ]
}

And the resulting model is still:

class Model(BaseModel):
    __root__: Union[int, Dict[str, Any]]

Maybe I should open another issue instead of spamming here? :)

[koxudaxi]

@m-kus

Maybe I should open another issue instead of spamming here? :)

Yes, I recommend opening a new issue.

I'm refactoring the code.
I try your schema. And I get this code.

# generated by datamodel-codegen:
#   filename:  a.json
#   timestamp: 2021-04-19T15:07:46+00:00

from __future__ import annotations

from typing import Dict, Union

from pydantic import BaseModel, Extra


class TokenInfo(BaseModel):
    class Config:
        extra = Extra.allow

    __root__: str


class Model1(BaseModel):
    class Config:
        extra = Extra.allow

    token_id: str
    token_info: Dict[str, TokenInfo]


class Model(BaseModel):
    __root__: Union[int, Dict[str, Model1]]

[koxudaxi]

I'm thinking whether TokenInfo should be str 🤔
token_info: Dict[str, str]

[koxudaxi]

The model has additionalProperties is false.
But, extra is allow 😢

class Model1(BaseModel):
    class Config:
        extra = Extra.allow

    token_id: str
    token_info: Dict[str, TokenInfo]

[koxudaxi]

@m-kus @kmorris81
I have released fixed version 0.11.0
Would you please check it?

[kmorris81]

Definitely a marked improvement! Thanks for the follow-up and quick turnaround time on this!

[m-kus]

@koxudaxi works like a charm, kudos and big thanks for the fixes!