Fastapi: Why does the `response_model` seem to `__init__` the same object twice?

[ginomempin]

First check

• [x] I added a very descriptive title to this issue.
• [x] I used the GitHub search to find a similar issue and didn't find it.
• [x] I searched the FastAPI documentation, with the integrated search.
• [x] I already searched in Google "How to X in FastAPI" and didn't find any information.
• [x] I already read and followed all the tutorial in the docs and didn't find an answer.
• [x] I already checked if it is not related to FastAPI but to Pydantic.
• [x] I already checked if it is not related to FastAPI but to Swagger UI.
• [x] I already checked if it is not related to FastAPI but to ReDoc.
• [x] After submitting this, I commit to one of:
  
  
  
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  
  
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  
  
  • Implement a Pull Request for a confirmed bug.
  
  

Description + Example

I have an endpoint that returns an instance of this Container class, which has a list as one of its attributes, called items.

import logging
from pydantic import BaseModel, Field
from typing import List

class Container(BaseModel):
    items: List[int] = Field(default_factory=list)

    def __init__(self, **data):
        logging.debug(f'Init-ing Container with {data}')
        super().__init__(**data)

        # At runtime, compute some value and add to the list
        self.items.append(99)

As shown, I overrode the __init__ because I have to "internally" modify the list after instantiation (same as in the Private Model Attributes section of the Pydantic docs). I'm also using the Field(default_factory=list) to make sure each instance creates its own list.

Testing the class on its own works as expected:

container_1 = Container(items=[1, 2, 3])
logging.debug(container_1.items)
container_2 = Container(items=[4, 5, 6])
logging.debug(container_2.items)

logging.debug(f'{id(container_1.items)}, {id(container_2.items)}')
assert id(container_1.items) != id(container_2.items)

```shell
[2020-11-17 08:58:24.713] root:DEBUG Init-ing Container with {'items': [1, 2, 3]}
[2020-11-17 08:58:24.713] root:DEBUG [1, 2, 3, 99]
[2020-11-17 08:58:24.713] root:DEBUG Init-ing Container with {'items': [4, 5, 6]}
[2020-11-17 08:58:24.713] root:DEBUG [4, 5, 6, 99]
[2020-11-17 09:43:22.566] root:DEBUG 4481341504, 4481418560

Now, I set it as a [response_model](https://fastapi.tiangolo.com/tutorial/response-model/) to one of my endpoints:

```python
@api.get('/things', response_model=Container)
def get_things():
    return Container(items=[1, 2, 3])

The response I get however has a duplicate 99 value:

This seems to indicate that the __init__ was called twice on the same object? It's somehow related to the Mutable Default Argument problem. Hitting the endpoint multiple times does not keep adding 99, so it seems it is still partially correct that it instantiates a new object, but seems to call __init__ twice.

[2020-11-17 09:23:08.501] root:DEBUG    Init-ing Container with {'items': [1, 2, 3]}
[2020-11-17 09:23:08.502] root:DEBUG    Init-ing Container with {'items': [1, 2, 3, 99]}

But testing the class on its own does not have this issue, and my endpoint is instantiating the class only once here. So, what am I doing wrong here?

Now, if I use instead the responses parameter:

@api.get('/things', responses={200: {'model': Container}})
def get_things():
    return Container(items=[1, 2, 3])

I get the correct response:

And the logs correctly show only one instantiation:

[2020-11-17 09:23:59.786] root:DEBUG    Init-ing Container with {'items': [1, 2, 3]}

So, what is wrong with my usage of the response_model parameter?

As for the Swagger/OpenAPI schema, using either response_model or responses show the current schema in the docs, so no problem/difference there. Also, I'm actually not sure if this is a Pydantic or FastAPI issue, but using just the class _outside of an endpoint_ that does not show this problem. So I figure, it's related to being used as a response_model.

Environment

• OS: macOS 10.15.7
• FastAPI Version: 0.61.2
• Pydantic Version: 1.7.2
• Python Version: 3.8.6

[Mause]

Fastapi internally initialises the model to validate the data that you are returning - this makes more sense in the case that you return a dictionary to transform into your model (and validated at the same time)

[ginomempin]

Fastapi internally initialises the model to validate the data that you are returning

Ok, and this only happens with the response_model ?

I tried to follow the code and with a response_model, it leads to a create_cloned_field:

           # Create a clone of the field, so that a Pydantic submodel is not returned
           # as is just because it's an instance of a subclass of a more limited class
           # e.g. UserInDB (containing hashed_password) could be a subclass of User
           # that doesn't have the hashed_password. But because it's a subclass, it
           # would pass the validation and be returned as is.
           # By being a new field, no inheritance will be passed as is. A new model
           # will be always created.
           self.secure_cloned_response_field: Optional[
               ModelField
           ] = create_cloned_field(self.response_field)

This is then passed to the route handler later on:

        self.app = request_response(self.get_route_handler())

    def get_route_handler(self) -> Callable:
        return get_request_handler(
            ...
            response_field=self.secure_cloned_response_field,

So I thought it was creating a separate instance, which should not have problems... right? Because creating instances with just the class itself (as I've shown) has no problems.

How then do I handle the case where the model has a mutable sequence type?

[Mause]

That only occurs with response_model yes, it's basically the point of that parameter.

A model mutating itself like that is a bit of a code smell, you probably want a separate model (that your mutating one can inherit from) to put in the response_model parameter

See the documentation for further details: https://fastapi.tiangolo.com/tutorial/response-model/

[ycd]

This is something expected and the point of the ResponseModel, it ensures your data is in the correct order.

This is actually an old topic, see the related issue #1359, and the related PR to add response_model_skip_validation #1434.

If you really need performance you can use ORJSONResponse which will return the data without any validation etc. See the documentation of Returning a response directly

[ginomempin]

The performance is not an issue, and while skipping validation looks like a viable workaround, I don't want to skip validation of the output response (the validation is also not the issue).

The behavior just looks "weird" to me that the model's __init__ code seems to be called again during validation.

For now, I've resorted to accepting Mause's comment that my current implementation of the model looks like a code smell, and I should not be mutating the model "internally" in __init__.